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CHAPTER  I 

INTRODUCTION 

Statement  of  the  Problem 

The  Air  Force  needs  an  improved  technical  order  (TO) 
format  (15:7).  The  recent  development  of  Logic  Tree  Trouble- 
shooting Aids  (LTTAs) , Fault  Isolation  Manuals  (FIMs) , 

Fault  Reporting  Manuals  (FRMs) , Job  Guide  Manuals  (JGMs) , 
and  System  Schematics  Manuals  (SSMs)  is  a major  improvement 
in  the  quality  of  weapon  system  TOs  (19:1-3).  However, 
an  analysis  of  the  additional  cost  of  procurement  versus 
the  potential  savings  generated  by  these  TO  improvements 
is  required  (19:48).  Therefore,  the  purpose  of  this  thesis 
is  to  determine  if  the  economic  benefits  derived  from  these 
new  developments  in  the  TO  format  exceed  the  higher  procure- 
ment cost. 

Justification 

For  many  years,  there  has  been  stronq  interest  in 
the  conservation  of  Department  of  Defense  (DOD)  funds  within 
the  United  States  Air  Force  (USAF) . Personnel  costs  have 
been  taking  an  increasingly  larger  percentage  of  the  overall 
Air  Force  budget  which  limits  the  funds  available  for  other 
Air  Force  programs  (26:150).  Recent  studies  have  indicated 
that  manpower  costs  for  maintenance  personnel  alone  account 
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for  15  percent  of  the  Air  Force  budget  (18:2).  If  the 
level  of  operations  and  effectiveness  is  to  ue  maintained 
in  the  USAF,  new  ways  of  improving  the  effectiveness  of 
its  personnel  must,  be  found  (24:2). 

The  complexity  of  present  Air  Force  weapon  systems 
and  the  steadily  increasing  cost  of  maintaining  these  sys- 
tems, despite  current  austerity  programs,  require  that  new 
ways  of  reducing  maintenance  costs  be  identified  and  imple- 
mented (18:1).  "One  way  of  reducing  cost  is  to  increase 
the  productivity  of  maintenance  personnel  by  providing 
better  technical  data  (18:1]."  The  desired  result  is 
reduced  maintenance  manhours,  improved  quality  of  the  prod- 
uct, increased  weapon  system  availability,  and  lower 
operating  cost. 

The  results  of  4 Study  oj'  the:  Air  Force  Maintenance 
Technical  Data  System,  accomplished  in  1962,  by  the  Behav- 
ioral Sciences  Laboratory  emphasized  the  problem  of  the 
increased  complexity  of  weapon  systems  and  the  growing 
number  of  TOs  required  to  support  them  (2:44).  According 
to  a 1974  estimate,  the  83,609  USAF  TOs  had  875,451,189 
pages  (27:A7-6).  "In  1940  the  average  aircraft  required 
approximately  1000  pages:  while  today,  the  average  air- 
craft requires  53,000  pages  [27:p.l-l]."  A weapon  system 
similar  to  the  roposed  B-l  would  require  a TO  system 
approaching  one  million  pages  (27:p.l-l).  The  rapidly 
increasing  volume  of  technical  data  needed  to  support 
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advanced  weapon  systems  makes  it  extremely  difficult  to 
effectively  utilize  the  information  available. 

The  acquisition  cost  of  the  increasing  volume  of 
technical  data  is  on  the  increase  (19:1).  The  F-15  Techni- 
cal Order  Management  Agency  (TOMA)  has  a budget  approaching 
$100  million  with  an  expenditure  rate  of  $1.1  million  per 
month  (36).  The  F-16  TOMA  has  a budget  of  $167.1  million 
with  an  expenditure  of  $1.8  to  $3  million  per  month  (22). 
A-10  TOMA  has  an  expenditure  rate  of  $.5  million  per  month 
with  a total  budget  ceiling  of  $20  million  (9) . The  enor- 
mous budgets,  expenditure  rates,  and  the  potential  finan- 
cial impact  of  procuring  and  implementing  new  types  of 
technical  data  make  it  imperative  that  the  decision  makers 
have  accurate  information  available  on  the  cost  and  bene- 
fits (19:13). 

The  development  of  more  usable  TO  systems  is  desir- 
able, and  the  technological  capability  required  to  develop 
improved  TO  systems  is  currently  available.  The  conven- 
tional TO  (CTO)  system  format,  MIL-M-25098,  does  not  meet 
the  operational  requirements  of  the  Air  Force  in  terms  of 
responsiveness,  accuracy,  and  flexibility  (2:22-23).  The 
present  system  is  outdated, and  it  fails  to  take  advantage 
of  technological  developments  in  providing  a means  of 
readily  identifying  and  displaying  the  data  necessary  to 
accomplish  individual  maintenance  actions  (31:1).  Products 
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unresponsive,  confusing,  and  incomplete  (36:2). 
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A 1974  draft  of  a Required  Operational  Capability 
(ROC)  indicated  that  the  TO  system  was  outdated  because  it: 

. . . failed  to  take  advantage  of  technological 
developments  in  providing  a means  of  rapidly  identi- 
fying and  retrieving  the  data  necessary  to  accomplish 
individual  maintenance  actions.  There  is  an  urgent 
need  for  a new  technical  order  system  which  is  more 
responsive  to  users  [<»:'.♦]  needs  [31:11. 

At  the  level  where  actual  maintenance  is  performed, 
the  use  of  systems-oriented  technical  data  will  help  ensure 
maximum  ut il  i zat ion  of  maintenance  personnel,  which  will 
improve  efficiency  and  reduce  cost  (18:2).  In  1976,  the 
Systems  Research  Laboratories,  under  contract  to  the  Human 
Rcsoucos  Laboratory,  evaluated  the  effectiveness  of  various 
types  of  TOs  used  to  troubleshoot  system  malfunctions. 

The  results  of  the  experiment  evaluation  clearly 
demonstrate  that  the  use  of  proceduralized  trouble- 
shooting [logic  tree]  approach  led  to  significantly 
better  troubleshoot  mg  than  the  use  of  the  [conven- 
tional) TO.  This  finding  is  consistent  for  . . . 
proportion  of  problems  solved  and  spares  consumed. 
Nearly  twice  as  many  good  parts  were  unnecessarily 
replaced  when  the  TO  was  used  than  when  LTTAS  ILogic 
Tree  Troubleshooting  Aids]  were  used  [19:86]. 

In  summary,  high  maintenance  personnel  cost  makes 
it  imperative  that  a more  efficient,  improved  TO  format, 
such  as  the  LTTA,  be  utilized.  Additionally,  the  increas- 
ing number  of  weapon  systems  and  their  complexity  demands 
that  a system  within  the  capabilities  of  present  technology 
be  established.  For  these  reasons,  the  l‘»76  study  sponsored 
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by  the  Air  Force  Human  Resources  Laboratory  (AFHRL) 
recommended  that  consideration  be  given  to  the  develop- 
ment of  LTTAs  for  existing  and  future  weapon  systems 
(21:17) . 

At  the  present  time,  LTTAs  are  being  procured  for 
the  F-16  aircraft  as  part  of  the  original  TO  information 
based  under  MIL-M-83495  (22) . The  technical  data  procured 
under  this  military  specification  also  includes  FIMs, 

FRMs,  JGMs , and  SSMs;  and  the  complete  package  is  refer- 
red to  in  this  study  as  proceduralized  technical  orders 
(PTOs) . Due  to  the  significant  advantages  of  PTOs,  the 
F-15  TOMA  manager  is  in  the  process  of  evaluating  the  cost 
of  procuring  the  PTOs  for  the  projected  F-15  C/D  aircraft. 
An  analysis  of  the  cost  versus  the  advantages  is  required 
to  assist  top  level  management  in  deciding  the  appro- 
priate type  TO  option  to  procure. 

Scope 

This  thesis  is  limited  to  analyzing  the  cost  versus 
the  benefits  of  procuring  technical  orders  (TOs)  in  the 
new  proceduralized  technical  order  (PTO)  format,  MXL-M- 
8345,  for  the  F-15  series  aircraft.  However,  the  con- 
clusions of  this  study  may  be  applicable  to  the  TO  procure- 
ment of  similar  weapon  systems. 


Objectives 


The  objectives  of  this  thesis  are  to: 

1.  Identify  the  steps  in  the  TO  procurement 
process  for  new  weapon  systems. 

2.  Explain  the  technique  used  by  McDonnell- 
Douglas  Aircraft  Company  to  develop  cost  estimates  for  the 
F-15  weapon  system  TOs. 

3.  Describe  the  advantages  of  procuring  the  F-15 
weapon  system  TOs  in  the  PTO  format. 

4.  Estimate  where  possible  the  monetary  savings 
derived  from  the  advantages  of  the  improved  PTO  format. 

Research  Question 

The  following  research  question  was  established 
for  this  study:  Should  the  F-lb  ueapon  system  technical 
data  be  procured  in  the  FTO  (MIL-M-8'6495)  format? 

Overview 

The  following  chapters  are  arranged  in  a logical 
sequence  to  answer  the  research  question.  Chapter  II 
contains  a thorough  literature  review  of  technical  data 
systems  and  cost  estimating  techniques.  This  chapter 
also  contains  background  information  about  data  systems 
and  computer  applications  used  later  in  the  thesis. 

Chapter  III  describes  the  methodology  which  consists  of 
two  parts:  background  for  analysis  and  approach  to 
analysis.  Assumptions  and  limitations  that  apply  to  the 
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CHAPTER  II 

LITERATURE  REVIEW 

Introduction 

The  literature  review  consists  of  three  major 
areas.  The  first  is  a summary  of  the  various  studies 
conducted  to  determine  the  types  of  improvement  required 
in  the  TO  system.  The  second  area  gives  an  overview  of 
the  different  types  of  cost  estimation/analysis  and  the 
limitations  and  advantages  of  each  type.  The  third  por- 
tion describes  the  data  source  and  computer  applications 
used  in  this  thesis. 

Technical  Data  System 

A Study  of  the  Air  Force 
Maintenance  Technical 
Data  System 

In  1962  a comprehensive  study  identified  the  need 
to  improve  maintenance  and  material  information  in  the 
USAF.  Unfortunately,  the  TO  system  in  general  use  today 
is  essentially  the  same  as  the  system  which  was  identified 
as  being  obsolete  in  1962.  This  study  of  the  Air  Force 
maintenance  technical  data  system  recommended  immediate 
action  to  revise  and  improve  the  system  by  developing 
new  media  for  presentation  of  technical  data  (2:44). 
Development  of  a system  for  storing  and  retrieving 
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maintenance  data  requires  a highly  refined  coding  system 
to  identify  and  describe  each  discrete  task  and  procedure. 
"The  magnitude  of  this  task  further  suggests  that  develop- 
ment action  should  be  undertaken  at  the  earliest  possible 
date  [2:44)." 

It  would  seem  that  no  time  should  be  wasted  in 
addressing  ourselves  to  this  problem,  and  that  the 
solution  could  be  applied  to  sophisticated  equipment 
on  the  ground  as  well  as  vehicles  in  space  [2:44]. 

The  purpose  of  the  19c>2  study  was  to  evaluate 
management  and  operating  procedures  and  to  identify  weak- 
nesses in  content  and  in  utilization  of  Air  Force  mainte- 
nance technical  data  (2:1).  The  2,300  responses  to  a 
field  survey  of  nineteen  organizations  provided  a factual 
and  definitive  basis  for  the  identification  of  problem 
areas  and  deficiencies  in  Air  Force  maintenance  technical 
data  (2:14).  Organizations  surveyed  included  base  level 
maintenance  personnel,  depot  maintenance  technicians, 
technical  data  managers,  military  and  civilian  staff 
officers,  and  unit  commanders  (2:2).  There  was  general 
agreement  among  technical  data  managers  that  the  Air 
Force  had  not  effectively  developed  technical  data  within 
its  capability  based  on  the  requirements  of  the  individual 
technician  i2:10).  Questionnaire  responses  suggested 
"...  a need  for  changes  in  the  size,  structure  and 
content  of  TOs  to  make  them  more  useful  both  as  a train- 


ing text  and  as  a job  performance  aid  [2:1b)."  Furthermore, 
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. . it  was  apparent  that  lack  of  detailed,  accurate 
and  current  information  in  TOs  not  only  impedes  mainte- 


nance  but  retards  training  progress  [2:12]." 

In  order  to  simplify  the  task  of  locating  needed 
information  required  to  do  the  job,  a revised  TO  numbering 
and  indexing  system  is  essential  to  modernization. 

The  one  subject  concerning  TOs  on  which  the  main- 
tenance man  was  most  positive  and  voluble  pertained 
to  the  difficulties  encountered  in  finding  the  infor- 
mation required  to  do  the  job  [2:32]. 

The  average  respondent  indicated  that  he  spent  30  percent 
of  his  total  job  time  in  seeking  necessary  information 
in  the  TOs  (2:32).  Additionally,  30  percent  said  informa- 
tion was  very  difficult  to  find,  and  47  percent  felt  that 
there  must  be  a better  system  (2:32).  The  following  is 
a typical  response  by  maintenance  personnel:  "You  can't 
go  to  any  one  place  in  the  TO  and  find  out  all  about  how 
a system  or  component  works--you  get  a little  bit  here  and 
go  dig  out  a little  bit  somewhere  else  [2:22]."  Generally 
speaking: 

It  appears  to  the  research  team  that  a subject 
of  overriding  importance  to  effective  use  of  mainte- 
nance technical  data  was  identification  of  methods 
and  techniques  to  make  the  necessary  information  for 
specific  tasks  more  readily  available,  and  easier  to 
identify  and  find  [2:33]. 

Respondents  to  the  questionnaire  identified  the 
following  categories  of  deficiencies  or  complaints  in  the 
physical  aspects  of  technical  aata:  (1)  the  numbering 

system  created  difficulty  in  finding  required  information; 
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(2)  the  amount  of  refer-backs  in  the  textual  procedures 
was  excessive;  (3)  large  numbers  of  TOs  were  required  to 
do  a single  job;  (4)  the  TOs  contained  inadequate  or  incom- 
plete  troubleshooting  information;  (5)  the  size  and  weight 
of  data  used  was  excessive;  and  (6)  the  data  were  late, 
inaccurate,  and  unrevised  (2:22-23). 

i 

s;  Maintenance  Information 

, Automated  Retrieval 

1 System  (MIARS ] 


The  Department  of  the  Navy  studied  the  feasibility 
of  converting  Navy  aeronautical  component  manuals  to  a 
Fault  Tree  Isolation  System  using  16mm  film  cartridges 
in  1967.  Their  objectives  were  very  similar  to  the  recom- 
mendations made  by  the  1962  Air  Force  study.  On  7 October 
1968  OPNAV  Instruction  4790.1  was  issued  by  the  Chief  of 
Naval  Operations.  The  purpose  of  this  instruction  was 
to  provide  additional  support  to  the  newly  established 
Maintenance  Information  Automated  Retrieval  System  (34:1). 
The  initial  goal  of  MIARS  was  to  utilize  the  16mm  film 
system  to  reduce  the  volume  of  TO  manuals  (35:11).  MIARS 
was  the  Navy's  answer  to  the  "...  ever-increasing  volume 
of  mainterance  and  material  information  [34:1]."  In  the 
last  five  years,  the  technical  base  for  all  first-line 
aircraft  weapon  systems,  such  as  the  A-7,  F-14,  and  P-3, 
has  been  set  up  in  the  logic  tree  format.  This  format 
provides  an  effective  means  of  retrieving  appropriate 
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troubleshooting  information  in  the  same  basic  style  as  the 
Air  Force's  Military  Specification  MIL-M-83495  (21). 

PIMP  Final  Report  Summary 

The  Presentation  of  Information  for  Maintenance  and 
Operation  (PIMO)  project  was  started  in  June  1966  and  com- 
pleted in  April  1969.  PIMO  studied  the  "...  complex  rela- 
tionship between  qualified  maintenance  manpower  and  techni- 
cal data  [23:2]."  Specifically,  the  test  attempted  to  prove 
that  a proceduralized  job  guide  technical  order  system  would 
reduce  maintenance  manhours  and  increase  the  reliability  of 
troubleshooting  for  apprentice  (three-level)  technicians 
(23:2)  . 

The  PIMO  field  study  was  conducted  on  the  C-141  air- 
craft at  Charleston,  Dover,  and  Norton  Air  Force  Bases.  It 
was  a follow-up  to  a 1965  study  made  by  Serendipity,  Inc., 
which  concluded  that  the  proceduralized  job  guide  concept 
would,  without  doubt,  improve  technician  performance 
(23:10).  When  evaluating  the  effectiveness  of  procedur- 
alized job  guides  on  more  complex  systems.  Serendipity  com- 
pared the  complexity  of  the  UH-1F  helicopter  to  the  C-141 
aircraft  and  stated  ".  . . that  the  improvement  in  system 
effectiveness  would  be  even  greater  for  complex  systems 
[23:3] . " 

The  PIMO  study  concluded  that  through  the  use  of 
job  guides  there  would  bo  a 50  to  100  percent  increase 
in  the  maintenance  manpower  availability  for  productive 


maintenance  labor.  The  reduction  in  maintenance  labor 


i 

* 


would  be  a direct  result  of  being  able  to  reduce  on-the- 
job  training  (OJT)  by  approximately  25  percent.  The  study 
concluded  that  unscheduled  maintenance  at  the  homo  base 
could  be  reduced  27  to  44  percent  and  that  the  opera- 
tionally ready  rate  would  increase  38  to  40  percent. 

This  equates  to  a reduction  of  30  to  39  percent  of  the 
manpower  required  for  unscheduled  maintenance  (23:15). 

It  was  estimated  that  through  just  the  reduction 
in  OJT  for  the  431X1E  technician,  procedural ized  job 
guides  would  save  (depending  on  the  life-time  of  the 
C-141A fleet)  22  to  100  times  ".  . . more  money  than  it 
would  cost  to  expand  the  job  guide  to  cover  the  entire 
C-141A  fleet  [23:15]."  The  job  guide  manuals  were  esti- 
mated to  be  capable  of  increasing  the  C-141A  fleet  effec- 
tiveness by  the  same  amount  as  adding  16.44  more  C-141A 
aircraft  to  the  fleet  (23:15).  The  PIMO  report  estimated 
that  the  benefits  of  a procedural ized  job  guide  manual  far 
outweigh  the  cost. 

Evaluation  of  C-141  Job  Guide 
Manuals  (JGM)  PRAM  Project 
1^475-62 

In  1976,  a study  was  conducted  to  ",  . . objec- 
tively evaluate  the  worth  of  the  already  procured  manuals 
in  an  operational  environment  [11:1]."  A comparison  was 
conducted  between  bases  which  utilized  JGMs  and  those 
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that  used  conventional  technical  orders  (CTOs) . The 
study  concluded  that  inexperienced  technicians  were  able 
to  perform  maintenance  tasks  using  the  JGMs  which  they 
were  unable  to  accomplish  using  the  CTO.  Additionally, 
it  was  concluded  that  JGMs  . . significantly  enhance 
the  proficiency  portion  of  the  on-the-job  training  (OJT) 
program  [11:2]." 

When  analyzing  the  "economic  pay-off"  the  study 
group  could  not  establish  a direct  cost  savings  (11:4-5), 
but  they  were  "...  confident  that  future  savings  will 
be  accrued  as  a result  of  the  data  and  the  effect  it  will 
have  on  existing  and  future  C-141  maintenance  technicians 
[11:5] ." 

Evaluation  of  Three  Types 
of  Technical  Data  for 
Troubleshooting 

The  Human  Resources  Laboratory  of  the  Air  Force 
Systems  Command  sponsored  a special  study  to  evaluate  dif- 
ferent types  of  technical  data  for  trouble  shooting  in 
1976.  This  study  evaluated  the  troubleshooting  effective- 
ness of  Fully  Procedural lzed  Troubleshooting  Aids 
(FPTAs) , Logic  Tree  Troubleshooting  Aids  (LTTAS) , 
and  Conventional  Technical  Orders  (CTOs) . The  study 
indicated  that  better  troubleshooting  was  accomplished 
using  either  the  FPTA  or  LTTA  than  the  CTO.  A greater 
proportion  of  the  problems  were  solved,  using  less  air- 
craft parts  when  the  maintenance  technicians  used  either 
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the  FPTA  or  the  LTTA  (19:86).  Additionally,  the  perfor- 
mance of  the  apprentice  technicians  using  FPTAs  and  LTTAs 
approached  the  performance  of  experienced  personnel. 
However,  because  of  the  extremely  high  cost,  the  FPTAs  are 
not  being  procured  at  this  time  (19:87). 

Technical  Data  Requirements 
for  Weapon  Systems-- 
14  June  1977 

Initial  Technical  Order  Project  Findings,  part 
one  of  a two-part  study  to  determine  the  cause  of 
".  . . the  alarming  increase  in  the  cost  of  technical 
orders  (TOs)  on  new  weapon  systems  [14:1],"  was  completed 
in  June  1977.  The  study  covered  five  major  areas: 
increases  in  TO  requirements,  TO  cost  drivers,  assess- 
ment of  user’s  requirements,  potential  TO  acquisition 
management  improvements,  and  other  recommendations.  The 
study  evaluated  the  four  areas  and  discussed  the  possible 
effect  the  elements  had  on  the  effectiveness  of  the  TO 
system.  The  study  concluded  that  "the  whole  TO  system 
cries  for  firmer  direction  and  control  . . . [29:38]." 

Therefore,  the  study  recommended  that  specific  actions 
should  be  taken  to  assure  that  quality  Air  Force  TOs 
are  acquired  in  the  most  cost  effective  manner  (29:2-9). 
The  following  recommendations/conclusions  are  those  that 
are  applicable  to  this  study: 

1.  Any  hope  of  effective  implementation  will 

require  additional  funds  as  well  as  high  level,  positive, 
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aggressive  action.  Modification  of  many  peoples'  attitudes 
and  behavior  is  required  for  successful  implementation  (29:4)  . 


2.  In  references  to  MIL-M-83495,  advocates  of  the 
latest  TO  military  specification  (AFHRL,  various  TO  coun- 
cil members)  claim  the  following  benefits  (29:23).  The 
procedural ized  Technical  Order  (PTO)  reduces  cost;  sim- 
plifies training/maintenance  actions;  uses  a numbering  sys- 

I 

tern  that  is  international  in  use;  bridges  the  gap  between 


designers,  engineers,  and  maintenance  personnel;  estab- 
lishes better  configuration  control  of  aircraft  wiring; 
and  establishes  an  integrated  organizational  maintenance 
specification  set  (29:23-24). 

3.  Additionally,  the  Job  Guide  Manuals  (JGMs) 


and/or  the  PTO  package  provide  the  following  benefits: 
technicians  can  perform  tasks  using  this  type  of  data  when 
they  are  unable  to  perform  the  same  tasks  using  conven- 
tional tech  data;  it  is  much  easier  for  first  term  airmen 
to  follow  and  use;  they  allow  productivity  sooner  with  less 
training;  they  reduce  time  in  search  and  retrieval  of  tech 
data;  they  reduce  maintenance  error  rate — removing  service- 
able components  during  troubleshooting;  they  can  be  used  by 
lower  skill  levels;  they  can  reduce  troubleshooting  time, 
OJT,  and  formal  training;  they  increase  maintenance  capa- 
bility; they  can  reduce  maintenance  manhours  per  flight 
hour;  they  improve  operational  readiness;  they  provide 
higher  in-commission  rates;  and  they  can  reduce  spare 
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parts  inventory  by  reducing  the  number  of  serviceable  parts 
erroneously  replaced  (29:24-25). 


Technical  Data  Requirements 
for  Weapon  Systems-- 
9 September  1977 

Part  two,  the  Final  Technical  Order  Project  Find- 
ings, was  completed  in  September  1977;  and  the  report 
is  prepared  in  three  major  sections:  F-15,  F-16,  and  C-141 
aircraft.  The  report  is  basically  a summary  of  the  most 
current  TO  cost  data  presently  available  for  the  three 
aircraft.  Each  section  addresses  TO  cost  data,  number  of 
TOs  and  number  of  pages;  and  a comparison  between  the 
initial  and  final  findings  is  made  for  the  three  specific 
aircraft.  Most  of  this  information  was  obtained  by  per- 
sonal interview  with  the  TOMA  managers.  The  results 
verify  two  observations  made  in  earlier  studies:  The  cost 
and  volume  of  modern  weapon  system  TOs  is  continuously 
increasing  (30:1). 


User  Acceptance  and  Usability 
of  the  C-141  Job  Guide 
Technical  Order  System 

Between  June  1975  and  February  1977  the  Human 
Resources  Laboratory  conducted  a study  to  determine  the 
latest  status  of  the  C-141  Job  Guide  TO  system.  Through 
the  use  of  questionnaires,  observations,  and  interviews, 
it  was  determined  that  the  program  was  successful  (15:1). 
It  was  felt  that  "The  JGMs  and  LTTAs  generally  have  been 


well  accepted,  although  some  resistance  to  change  was 
encountered.  The  new  technical  data  have  generally  been 
considered  to  be  superior  to  the  technical  orders  that 
they  replaced  [15:1]." 

Positive  and  negative  factors  were  observed.  The 
positive  included:  size  of  the  books,  clarity  and  organi- 
zation of  materials,  use  of  illustrations,  and  dual-level 
presentation  of  instructions.  Negative  factors  included: 
resistance  to  change,  errors  in  data,  too  many  volumes 
required  for  some  jobs,  lack  of  storage  space  for  the  TOs, 
easily  torn  pages,  method  of  locating  information  difficult 
to  master,  and  implementation  (15:1).  Additionally,  specific 
recommendations  for  possible  corrective  action  of  the  above 
noted  negative  observations  were  given. 


Cost  estimations,  no  matter  how  sophisticated  the 
estimating  process  used,  are  actually  only  educated  guesses. 
Government  agencies,  as  well  as  industry,  extrapolate  from 
historical  data  the  planning  estimates  required  to  answer 
the  questions  concerning  the  development  of  future  programs. 
Thus,  the  real  basis  for  estimating  future  costs  is  compari- 
son with  experienced  costs.  The  methods  for  making  cost 
comparisons  range  from  expert  opinion  down  to  detailed 
industrial  engineering  computations.  The  primary  differences 


18 


in  the  continuum  of  cost  estimating  methods  are  the  tech- 
niques used  to  extrapolate  from  the  known  to  the  unknown 
(7:45)  . 

In  general,  there  are  three  basic  approaches  to 
cost  estimating:  Analogous  Systems , Industrial  Engineering 
and  Parametria.  Each  of  these  approaches  to  cost  esti- 
mating is  frequently  used  in  the  preparation  of  cost  esti- 
mates during  the  acquisition  process.  The  strengths, 
weaknesses,  and  appropriateness  of  each  technique  are  dis- 
cussed in  the  following  paragraphs  (7:3). 

The  Analogous  System  approach  of  estimating  costs 
by  drawing  analogy  to  other  items  is  probably  the  most 
prevalent  methodology  used.  This  technique  is  a direct 
comparison  of  a new  program  or  program  component  to  one  or 
more  recent  and  similar  projects.  This  method  is  used  to 
obtain  a broad  cost  assessment  (ball  park  estimate)  of  a 
program  cost;  and  when  it  is  applied  in  a carefully  planned, 
detailed,  and  conscientious  manner,  analogy  is  perhaps  the 
most  powerful  method  of  estimating.  The  greatest  advantage 
derived  from  this  method  is  that  it  can  be  made  quickly 
without  the  time  and  cost  otherwise  required  to  develop 
an  in-depth  analysis.  Unfortunately,  the  analogous  system 
is  too  often  applied  in  a quick,  haphazard  manner;  there- 
fore, the  system  frequently  receives  a skeptical  reception. 
It  only  considers  technological  costs  if  the  analogous 
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system  used  for  comparison  accounted  for  those  costs. 
Ironically,  the  primary  disadvantage  of  analogous  estimates 
is  their  simplicity;  and  a great  advantage  is  the  same 
simple  ease  of  application.  However,  the  estimate  is 
sometimes  based  on  "guesses"  with  little  factual  data  to 
support  the  cost  of  new  components  which  are  different  from 
the  compared  system.  Analogy  comparison  often  tends  to  be 
less  acceptable  to  prospective  users  because  it  lacks 
statistical  tests  (7:4). 

Estimators  use  the  analogous  system  to  predict  the 
cost  of  the  unknown  items  by  comparing  them  to  items  that 
are  similar  in  function,  construction,  and  technology.  The 
final  estimate  is  strengthened  when  several  comparisons 
for  each  item  are  made.  Concentration  on  the  largest  cost 
elements  becomes  necessary,  and  determined  efforts  must 
be  used  to  comprehensively  document  the  entire  approach 
and  overcome  preconceived  notions  of  inadequacy  in  the 
analogy  approach.  After  a system  enters  the  conceptual 
phase,  the  Analogous  Sgsten  method  is  not  normally  used 


(7:4)  . 


The  Industrial  Engineering  approach  can  be  used 


when  the  item  design  is  well  known  and  well  established. 
This  approach  begins  with  an  analysis  of  the  work  proposed 
and  an  extensive  description  of  the  system  and  design 
requirements.  Manhours  (labor)  and  material  requirements 
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are  accumulated  and  costed,  and  then  the  elements  are 
summed  to  arrive  at  a total  cost  for  the  job.  Since  this 
method  uses  standards  built  from  time  and  motion  studies, 
Gantt  Chart  analysis,  critical-path  scheduling,  etc., 
detailed  production  operations  must  be  known.  After  the 
standards  are  established,  current  or  historical  cost 
factors  are  applied  to  the  analysis  (7:5). 

The  greatest  difficulty  in  applying  this  approach 
is  that  the  Air  Force  is  not  the  producer  of  technical  data; 
therefore,  the  Air  Force  does  not  have  direct  knowledge 
or  control  of  the  manufacturing  process.  Without  this 
knowledge  and  control  at  his  disposal,  the  estimator's 
only  alternative  is  to  use  the  limited  data  available  and 
personal  past  experience  to  roughly  price  the  cost  of 
materials,  labor,  overhead,  etc.  He  cannot,  however, 
estimate  direct  labor  or  overhead  costs  with  any  reliable 
degree  of  accuracy  since  these  factors  depend,  to  a great 
extent,  upon  the  manufacturing  process  and  accounting  pro- 
cedures utilized  by  the  individual  contractor.  Thus,  the 
Air  Force  estimator  must  rely  on  cost  factors  and  cost 
estimates  provided  by  the  contractor  because  there  are  no 
industry  standards  for  the  computation  of  technical  data 
costs.  Consequently,  if  the  contractor  does  not  submit 
accurate  cost  data,  the  estimator  must  either  develop 
general  factors  that  lump  direct  labor,  burden,  and 


21 


profit  together  or  must  apply  undocumented  "rule-of-thumb" 
factors  (7:5). 

The  engineering  method  of  cost  estimating  works 
best  when  the  production  configuration  is  known  and  most 
problems  have  been  solved.  It  has  definite  limitations 
early  in  the  weapon  system  development  cycle  because  of 
procedural  difficulties  in  handling  "unknowns."  However, 
the  Industrial  Engineering  method  represents  the  most  pre- 
cise approach  to  estimating,  and  it  is  the  basis  for  most 
production  contracts  (7:506). 

The  Parametria  approach  to  cost  estimating  pro- 
duces an  estimate  which  predicts  costs  based  on  relation- 
ships between  variables  such  as  performance  characteris- 
tics, physical  traits,  and  developmental  distinctions 
derived  from  experience  on  logically  related  systems.  The 
relationships  stem  from  cost  histories  of  similar  systems, 
thus  the  parametric  approach  evaluates  the  new  system 
in  light  of  past  experience.  This  method  seems  most  use- 
ful whenever  a new  item  is  not  totally  similar  to  the 
existing  items  in  all  aspects  (7:48). 

The  Analogous  System  and  the  Parametria  approach 
are  similar  because  they  generally  require  large  amounts 
of  the  same  kind  of  data.  The  data  must  include  specifi- 
cations and  costs  of  previously  purchased  items,  and  an 
understanding  of  all  the  conditions  under  which  the  items 


were  developed  and  produced  must  be  acquired.  Once  the  data 
are  defined,  ratios  derived  from  a number  of  procurements 
are  analyzed  to  determine  if  significant  statistical 
trends  can  be  identified.  After  putting  the  data  together, 
cost  is  expressed  in  terms  of  the  performance  and/or  physi- 
cal characteristics  obtained  earlier,  and  this  data  now 
form  Cost  Estimating  Relationships  (CERs)  (7:6). 

Analysis  is  then  required  to  establish  what  CERs 
are  valid  for  the  cost  estimate  being  developed.  The 
validity  of  the  estimate  will  depend  upon  the  relevance  of 
the  CERs  and  the  confidence  placed  on  the  trends.  The 
parametric  approach  also  includes  such  intangibles  as 
schedule  slippages  due  to  limitations  on  funds,  technical 
problems,  changes  in  production  rates,  contract  performance 
failures,  management  inefficiencies,  labor  strikes,  and 
other  unknown  factors  in  the  program.  Parametric  estimates 
should  be  logically  and  statistically  evaluated  rather 
than  routinely  accepted. 

The  parametric  estimating  approach  is  required  by 
DOD  Memorandum  for  all  cost  estimates  presented  to  the 
Defense  Systems  Acquisition  Review  Council  (DSARC) . Basic- 
cally,  this  approach  permits  a blending  of  known  changes 
in  system  acquisition  management  and  technology  with 
the  uncertainties  of  system  design  during  the  early  devel- 
opment phase  of  the  acquisition  process.  The  Parametric 
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estimate  (periodically  updated  by  known  changes  in  manage- 
ment, technology,  and  data)  can  be  used  as  a check  on  the 
more  definitive  Industrial  Engineering  cost  estimate  (7:7). 


Purposes  of  Cost 
Estimating/ Analysis 

Cost  estimates  are  required  for  three  general 
purposes:  Planning,  Budgeting,  and  Contracting. 

These  purposes  should  represent  milestones  in  a 
continuous  estimating  process  [7:7]. 

For  planning  cost  estimates,  detailed  cost  analyses  are 

not  generally  feasible  or  essential;  therefore,  analogous 

or  parametric  estimating  techniques  are  usually  employed. 

The  estimates  are  rough  but,  in  most  instances, 
sufficient  since  the  lack  of  a satisfactory  base  for 
accurate  estimating  in  the  planning  stage  dictates 
the  shunning  of  detailed  techniques  [7:7]. 

Rules-of-thumb  developed  by  the  parametric  method  can  be 

competently  applied  at  this  early  stage  in  many  cases 


(7:7)  . 


After  a decision  to  go  forward  with  a program  has 


been  made,  a budget  estimate  must  be  prepared.  The  budget 
computation  should  reflect  the  best  estimate  of  the  total 
program  at  the  date  of  procurement.  Since  the  budget  prep- 
aration process  precedes  contracting  by  at  least  a year, 
estimating  errors  caused  by  the  time  factor  and  vague 
requirements  are  introduced  at  this  point. 

The  contract  cost  estimating  process  normally 
includes  an  Independent  Government  Cost  Estimate  prepared 
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prior  to  solicitation  and  a Government  Contract  Negotia- 
tion Objective  resulting  from  an  analysis  of  proposals  in 
response  to  the  solicitation  (7:8). 

Obviously,  the  cost  estimating  process  is  continuous 
from  the  initial  planning  to  the  solicitation  of  proposals 
and  final  awarding  of  a contract. 

The  lack  of  an  adequate  data  base  for  accurate 
estimating  in  the  planning  stage  dictates  that  as  the  pro- 
gram progresses  into  the  acquisition  process,  new  data  will 
inevitably  become  available  which,  in  turn,  lead  to  changes, 
refinements,  and  more  realistic  cost  estimates.  Therefore, 
until  better  estimating  techniques  become  available,  the 
estimator  and  the  Air  Force  must  recognize  that  early 


estimates  are  rarely  meaningful  and  must  proceed  with  the 
development  of  procurement  based  on  that  premise  (7:8). 

Cost  Estimating  Uncertainties 

"Whenever  an  analyst  estimates  the  cost  of  a new 
system,  he  encounters  a multitude  of  problem  areas  which 
introduce  uncertainty  in  his  estimate  [7:9].”  Uncertainty 
occurs,  and  it  plays  a vital  role  even  in  the  seemingly 
most  simple  tasks. 

"Making  useful  cost  estimates  of  future  programs 


is  no  easy  task  [7:10]."  Thus,  it  can  be  safely  stated 
that  uncertainties  cannot  be  accurately  forecasted,  and 
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provisions  must  be  made  for  unforeseen  situations.  A risk 
situation  is  one  in  which  the  outcome  is  an  uncontrollable 
random  event  stemming  from  a knoun  probability  distri- 
bution; on  the  other  hand,  an  uncertain  situation  is 
characterized  by  the  fact  that  the  probability  distribution 
of  the  uncontrollable  random  event  is  unknown.  "This  dis- 
tinction, sometimes  leads  analysts  to  describe  risks  as 
'known-unknowns'  and  uncertainties  as  'unknown-unknowns' 
[7:11]  . " 

There  are  many  types  of  methods  that  the  cost  esti- 
mator may  turn  to  when  dealing  with  uncertainty.  The  fol- 
lowing list  is  only  a representative  sample  of  the  techniques 
used  to  manage  uncertainty:  Monte  Carlo  Simulation; 
Fortiori  Analysis;  Sensitivity  Analysis;  Range  of  Estimates 
Approach;  Supplemental  Discounting;  Adjustment  Factors;  and 
Special  Studies  (7:11-13).  The  successful  cost  estimator 
must  plan  for  unknowns,  or  his  estimates  will  not  be  valid 
in  a "Real  World"  which  is  filled  with  uncertainties. 


Data  Source  and  Computer  Techniques 

Increase  Reliability  of 
Operation  Systems  (IROS) 

The  Increase  Reliability  of  Operational  Systems 

(IROS)  uses  the  Industrial  Engineering  approach  to  estimate 

".  . . logistic  support  costs,  systems  downtime,  flight 

safely  [sic]  data  in  quantitative  displays  such  as  rank 
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orders  for  technical  managers  to  use  in  decisions  on  where 
and  how  to  improve  system  effectiveness  [28:2]."  Implemen- 
tation of  IROS  is  in  accordance  with  Air  Force  Regulations 
400-46  and  Air  Force  Logistics  Command  Regulation  400-16. 
IROS  data  provides  a rank  ordering  of  information  for  each 
component  down  to  the  lowest  work  unit  code  (WUC)  level 
within  each  weapon  system. 

The  IROS  data  are  generated  quarterly  in  the  Air 
Force  K05 1 data  system.  The  information  is  collected  daily 
from  Air  Force  Manual  66-1  Maintenance  Data  Collection 
System,  the  Air  Force  Manual  65-110  Equipment  Status  Report- 
ing System,  and  various  depot  level  repair  management  sys- 
tems (13:2).  "IROS  transforms  day  to  day  logistics  activi- 
ties reported  in  support  of  a weapon  system  to  support 
dollars  in  terms  of  BASE  LABOR,  DEPOT  LABOR,  DEPOT  MATERIAL, 
COST  OF  CONDEMNATIONS,  TRANSPORTATION,  AND  PACK/SHIP  COST 
[28:2] . " 

SIMFIT 

The  SIMFIT  computer  program  tests  the  distribution 
of  raw  data  against  twelve  distributions  contained  within 
the  program.  These  distributions  are:  Erlang;  Weibull; 
Gamma;  Pearson  XI;  Lognormal;  Normal;  Uniform;  Beta; 
Triangular;  Poisson;  Binomial;  and  Negative  Binomial. 
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The  SIMFIT  program  calculates  the  parameters  needed 
to  determine  the  probability  distribution  from  the  input 
data.  The  input  data  are  divided  into  cells  and  com- 
pared to  the  theoretical  value  for  each  cell  [6:85]. 

The  computer  program  uses  the  Kolmogorov-Smirnov 
(K-S)  and  the  Chi-square  goodness-of-fit  tests.  However, 
the  K-S  test  is  the  only  one  that  can  be  used  when  the 
sample  size  is  relatively  small  because  the  Chi-square  sta- 
tistics are  only  computed  for  that  portion  of  the  distribu- 
tion in  which  the  cell  size  criteria  are  met  (32:14). 

Monte  Carlo  Technique 

The  Monte  Carlo  technique  has  been  defined  as 
".  . . that  branch  of  experimental  mathematics  which  is 
concerned  with  experiments  on  random  numbers  [12:2]." 
Although  random  numbers  and  random  processes  have  been 
used  for  more  than  250  years,  the  Monte  Carlo  simulation 
is  a relatively  new  technique.  Basically,  the  Monte 
Carlo  simulation  is  a technique  to  ".  . . determine  some 
probabilistic  property  of  a population  of  objects  or 
events  by  the  use  of  random  sampling  applied  to  the  compo- 
nents of  the  objects  or  events  [1:175]." 

In  most  cost  estimating  the  probabilistic  property 
of  total  cost  is  determined  by  using  random  samples  of  the 
components  which  comprise  the  total  cost.  Such  applica- 
tions of  the  Monte  Carlo  technique  have  been  used  so  often 
in  simulation  models  that  the  term  Monte  Carlo  has  almost 
become  synonymous  with  the  word  simulation. 
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Summary 

The  problem  of  inadequate  technical  orders  was 
identified  over  fifteen  years  ago.  Additional  studies 
recently  conducted  by  AFHRL  and  AFLMC  have  reconfirmed  the 
observations  made  in  1962.  The  need  for  an  improved  TO 
format  in  terms  of  size,  structure,  content,  and  detail 
is  well  documented.  In  1976,  the  Human  Resources  Labora- 
tory field  tested  two  new  formats  which  had  been  developed 
for  the  purpose  of  correcting  the  well  documented  deficien- 
cies in  the  present  TO  format.  The  increased  cost  and 
value  of  TOs  for  modern  weapon  systems  makes  it  essential 
that  the  cost  of  converting  to  a new  TO  format  be  compared 
to  the  potential  savings  resulting  from  the  advantages 
provided  by  that  system. 

In  general,  three  basic  approaches  to  cost  esti- 
mating are  in  common  use  in  the  Air  Force  today:  Analogous 
Systems,  Industrial  Engineering,  and  Parametric.  Each 
of  these  approaches  to  cost  estimating  plays  an  important 
part  in  the  preparation  of  cost  estimates  during  the  acqui- 
sition process.  These  estimates  are  required  for  the  pur- 
pose of  planning,  budgeting,  and  contracting;  and  each  of 
these  purposes  represents  a milestone  in  the  continuous 
estimating  process.  Uncertainty  is  inherent  to  estimating, 
so  the  estimator  must  develop  a plan  to  manage  the  "unknown- 


unknowns. 
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IROS  transforms  day-to-day  logistics  activities 
into  data  which  can  be  stored  and  used  to  predict  future 
support  costs  for  a weapon  system.  Computer  techniques 
such  as  SIMFIT  and  Monte  Carlo  simulation  are  useful 
methods  of  analyzing  and  applying  the  IROS  data  to  cost 
estimating  problems. 
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CHAPTER  III 


METHODOLOGY 


Introduction 


This  thesis  methodology  contains  two  major  areas. 
The  first,  "Background  for  Analysis"  includes  a defini- 
tion and  discussion  of  the  nature,  sources,  and  collection 
of  the  data;  criteria  for  research  and  comparison;  and 
assumptions  and  limitations.  The  second  area,  "Approach 
to  Analysis"  encompasses:  the  sequence  of  steps  in  TO 
procurement;  the  TO  cost  estimation  method;  an  evaluation 
of  proceduralized  TOs  (PTOs) ; and  the  forecast  of  F-15 
logistics  cost. 


Background  for  Analysis 


Nature,  Sources,  and 
Collection  of  Data 


Data  for  TO  procurement  are  based  upon  historical 
records  of  past  procurements  from  the  same  contractor. 

For  example,  the  F-16  budget  information  is  based  upon 
data  obtained  from  General  Dynamics  during  the  development 
and  acquisition  of  the  F-lll  (26) . The  actual  cost  of 
various  types  of  TOs  is  based  on  the  contractor's  esti- 
mate of  the  average  number  of  manhours  required  to  produce 
one  page  of  the  TO.  The  various  categories  of  TOs  are 
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assigned  different  manhours  per  page  depending  on  the  com 
plexity  of  the  manual. 

The  cost  of  any  given  TO  is  largely  a contractor 
derived  product,  usually  computed  on  the  contractor's 
estimate  of  the  number  of  original  and  change  pages 
in  the  TO,  hours  required  to  prepare  those  pages  and 
cost  per  hour.  These  estimates  in  turn  may  be  based 
on  a variety  of  factors  such  as  contractor  developed 
page  and  illustration  standards  (hours  per  unit)  for 
writing,  graphics  and  production;  historical  data; 
surveys  of  existing  TOs;  complexity  of  the  system; 
and,  of  course,  interpretation  of  the  applicable  MIL 
Specs  (28:32-33] . 

The  validity  of  the  contractor's  estimates  is  nor 
mally  established  by  comparing  that  company's  previous 
performance  on  similar  weapon  systems  and  similar  manuals 
Additionally,  the  contractor's  claims  are  compared  to 
other  contractors  within  the  aerospace  industry  (22) . 

The  sequence  of  steps  in  the  TO  acquisition  were 
extracted  from  a comprehensive  literature  review.  In 
addition,  many  telephone  and  personal  interviews  were  con- 
ducted with  personnel  in  the  technical  data  business 
including  the  Air  Staff;  the  Air  Force  Plant  Representa- 
tive Office  (AFPRO) ; the  Air  Force  Logistics  Management 
Center  (AFLMC) ; the  Air  Force  Logistics  Command  (AFLC) ; 
the  A10,  F-15,  and  F-16  Systems  Program  Offices  (SPOs) ; 
and  the  Air  Force  Human  Resources  Laboratory  ( AFHRL) . 

The  data  on  the  advantages  and  disadvantages  of 
PTOs  were  primarily  based  on  a study  initiated  by  the 
Advanced  Logistics  Division,  Air  Force  Human  Resources 
Laboratories,  Inc . , Dayton,  Ohio. 
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Two  electronic  subsystems  of  the  C-141  aircraft  were 
used  as  the  foundation  for  the  test.  The  systems  selected 
were  the  AN/APN-147  and  the  AN/ASN-35  which  are  both  part 
of  the  doppler  radar.  They  were  chosen  for  the  following 
reasons:  the  equipment  was  readily  available;  the  CTOs  were 

representative  af  current  technical  data;  the  Air  Training 
Command  Course  3ABR32834  (Inertial  and  Radar  Navigation 
System  Specialist)  addresses  these  systems  as  "typical" 
types  of  Air  Force  equipment;  and  the  systems  were  complex 
enough  to  provide  a realistic  test,  but  small  enough  to  be 
able  to  procure  the  technical  data  at  a reasonable  cost 
(19:4,24) . 

The  identification  of  the  support  costs  for  the  F-15 
weapon  system  was  based  on  the  most  current  data  found  in 
the  K051  data  base.  These  data  are  based  on  quarterly 
information  from  the  AFR  66-1  Maintenance  Data  Collection 
System  (MDC) . The  K051  is  a "high  burner  list"  which  tracks 
the  scheduled  and  unscheduled  general  support  cost  of  the 
various  components  and  end  items  of  a weapon  system.  This 
tracking  system  represents  a key  part  of  the  Air  Force 
Logistic  Command's  (AFLC’s)  Material  Improvement  Program  and 
provides  specific  data  for  the  Increase  Reliability  of 
Operational  Systems  (IROS)  program  (8) . 

The  K051  data,  displayed  on  16mm  microfilm  and 
microfiche,  break  down  the  support  costs  of  a weapon  system 
by  the  Work  Unit  Code  (WUC) . The  cost  indicated  for  each 
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WUC  is  generated  at  the  organizational  and  intermediate 
base  level  maintenance  function  (8) . 


Criteria  for  Research 
and  Comparison 

The  Sequence  of  Steps  in  TO  Procurement  were  ana- 
lyzed with  the  intent  of  making  them  as  factual,  accurate, 
and  comprehensible  as  possible.  The  steps  were  written 
with  emphasis  on  the  cost  aspect;  thus,  any  omissions  or 
errors  that  one  might  encounter  were  due  to  the  cost  orien- 
tation of  this  study. 

Likewise,  TO  Cost  Estimating  Methods,  were  devel- 
oped with  the  intent  of  accurately  and  factually  reflecting 
the  actual  methods  currently  in  use  by  contractors  and 
Air  Force  experts  in  the  field.  As  with  the  steps  in  the 
TO  procurement  process,  the  cost  methods  were  obtained  from 
those  indiviudals  recognized  as  experts  in  the  procurement 
and  management  of  the  Air  Force  technical  data  systems. 

The  ultimate  goal  was  to  portray  the  real  world  in  terms 
of  TO  cost  estimating. 

The  Evaluation  of  the  PTOs  was  conducted  with  two 
primary  questions  in  mind;  What  advantages  and  disadvan- 
tages of  PTOs  have  an  impact  on  the  life  cycle  cost  of 
maintaining  the  weapon  system?  Can  realistic  present  day 
dollar  values  be  assigned  to  these  advantages  and  dis- 
advantages? In  some  cases  the  information  available  was 
too  subjective  to  assign  cost  figures.  Also,  some 
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conclusions  relevant  to  PTOs  used  on  the  C-141  aircraft 
are  not  applicable  to  the  F-15.  Consequently,  for  the  sake 
of  objectivity,  only  those  advantages  and  disadvantages 
that  were  quantifiable  and  analogous  to  the  F-15  were  used 
to  predict  the  dollars  saved  over  the  F-15  life  cycle. 

The  criteria  for  the  Cost  Analysis  of  the  Advan- 
tages and  Disadvantages  of  PTOs  hinged  on  the  nature  and 
characteristics  of  the  maintenance  data  obtained  from  the 
K051  data  base.  These  data  were  analyzed  to  determine  the 
extent  to  which  valid  application  of  these  data  could  be 
made  in  computing  life  cycle  dollar  values.  In  order  to 
maintain  the  objectivity  of  this  thesis,  the  cost  analysis 
portion  was  accomplished  based  on  the  K051  data  base  infor- 
mation. Again,  some  of  these  data  were  irrelevant  or 
incapable  of  quantitative  analysis  in  terms  of  cost.  Only 
the  data  which  satisfied  the  specific  objectives  of  this 
thesis  were  used  to  develop  the  life  cycle  cost  (LCC) . 

Assumptions  and  Limitations 

Prior  to  conducting  the  proposed  analysis,  some  basic 
assumptions  were  required  to  facilitate  the  research  and 
to  clarify  any  misinterpretations  that  the  reader  might 
perceive.  Therefore, the  following  assumptions/limitations 
were  established: 

1.  The  studies  conducted  by  project  PIMO,  the 
Systems  Research  laboratories/AFHRL,  and  AFLMC  are  valid 
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indications  ot  the  acceptability,  usability,  advantaqes, 
and  disadvantages  of  PTOs. 

2.  Advantaqes  and  disadvantages  of  PTOs  exist 
that  lend  themselves  to  quantitative  cost  analysis  by 
assigning  dollar  values  to  these  factors. 

3.  The  cost  estimates  for  procurement  of  various 
TO  options  for  the  F-15  weapon  system  which  were  provided 
by  McDonnell-Douglas  Aircraft  Corporation  are  accurate 
based  on  current  technology  and  experience  with  PTOs  and 
CTOs. 

4.  The  data  obtained  form  AFLC's  Maintenance 
Data  Collection  (MDC)  systems  are  reasonably  valid,  and 
these  data  can  be  used  to  place  realistic  dollar  values 

on  the  factors  identified  in  the  studies  conducted  on  PTOs. 

Approach  to  Analysis 

The  Sequence  of  Steps 
in  TO  Procurement 

The  exact  flow  of  the  TO  procurement  process  varies 
from  weapon  system  to  weapon  system.  A synthesis  of  the 
F-15  TO  procurement  process  will  be  made.  A diagram  show- 
ing the  sequence  will  be  established  using  a proceduralized 
step  method.  This  will  provide  a conceptual  and  visual 
display  of  the  technical  data  procurement  process. 
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TO  Cost  Estimation  Method 

Once  the  sequence  for  TO  procurement  has  been 
established,  it  will  be  possible  to  identify  the  various 
cost  elements  in  technical  data  costing;  such  as  manhours, 
pages,  and  category.  A decision  matrix  showing  the  alter- 
nate procurement  selection  possibilities  will  be  developed. 
This  matrix  will  show  the  alternative  decisions  and  the 


corresponding  cost  to  procure  the  technical  orders  for  the 

F-15. 


An  Evaluation  of 
Proceduralized  TOs  (PTOs) 

The  evaluation  of  the  PTOs  will  be  divided  into  the 
positive  and  negative  factors.  The  factors  will  be  defined, 
evaluated,  and  quantified  where  possible.  An  analysis  of 
the  various  effectiveness  and  efficiency  measurements  will 
be  accomplished  to  determine  the  overall  improvement 
in  maintenance  performance.  This  PTO  percentage  savings 
index  will  be  depicted  on  a scale  from  pessimistic  to 
optimistic  with  the  most  likely  value  being  the  midpoint  of 
the  two  extreme  points. 


Forecast  of  F-15 
Logistics  Cost 

The  operating  cost  of  the  F-15  will  be  categorized 
according  to  work  unit  codes  (WUCs)  using  the  last  three 
years  of  data  from  the  K051  data  base.  To  determine  the 
most  probable  cost  distribution  for  each  WUC,  three 
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techniques  will  be  used.  These  techniques  are:  time- 
data  graphs;  Simple  Linear  Regression;  and  the  SIMFIT 
computer  program.  Finally,  the  Monte  Carlo  cost  forecast- 
ing method  will  be  used  to  provide  the  probable  distribution 
of  the  total  logistics  cost  for  the  applicable  WUCs. 

Break-Even  Analysis 

A cost  benefit  analysis  of  the  PTO  will  be  made 
using  the  dollar  values  established  in  the  previous  sec- 
tion. Break-even  charts  will  be  used  to  indicate,  where 
possible,  at  what  point  in  time  the  additional  cost  of 
procuring  the  TOs  under  the  PTO  format  would  be  offset 
by  dollars  saved  using  PTOs.  The  same  logic  and  methodology 
will  be  extended  for  a portion  of  the  projected  life  cycle 
of  the  F-15  weapon  system  in  order  to  predict  the  total 
potential  savings  of  procuring  the  technical  data  base 
in  the  PTO  format. 
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CHAPTER  IV 


TECHNICAL  ORDER  PROCUREMENT 
PROCESS 

Procurement  Sequence 

The  acquisition  of  technical  orders  (TOs)  is  a 
complex  and  ever  changing  task  which  evolves  over  a period 
of  years.  The  exact  method  of  initially  procuring  TOs 
varies  from  weapon  system  to  weapon  system  and  is  controlled 
and  maintained  by  the  System  Project  Office  (SPO)  technical 
order  division. 

The  major  weapon  system  source  selection  process 
includes  the  requirement  for  the  contractor  to  provide  nega- 
tives of  the  applicable  technical  data  (based  on  the  mili- 
tary specification  stated  in  the  contract)  to  the  DOD 
representative.  The  DOD  representative  in  the  case  of  the 
F-15  is  the  Air  Force  Plant  Representative  Office  (APPRO) 
located  at  the  major  contractor's  plant  in  St.  Louis, 
Missouri.  The  AFPRO  assembles  the  negatives,  performs  a 
quality  control  check,  and  forwards  the  negatives  to  a 
contractor  designated  by  the  Government  Printing  Office 
(GPO) . The  technical  order  division  of  the  SPO  provides 
author 1 /at  ion  to  the  Air  Force  Systems  Command  (AFSC/DAR) 
who  m cum  obligates  the  funds  to  the  GPO  designated 
contractor  upon  completion  of  the  required  printing. 
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The  contractor  returns  the  negatives  to  the  APPRO  and 
distributes  the  formal  copies  of  the  technical  orders 
directly  to  the  USAF  field  units  through  the  TO  distribu- 
tion system.  However,  preliminary  technical  orders  are 
not  distributed  through  the  distribution  system  but  are  sent 
directly  to  the  SPO,  which  in  turn  distributes  them  as 
required.  The  APPRO  returns  the  negatives  to  the  weapon 
system  contractor,  who  is  responsible  for  maintaining  the 
negatives  until  program  transfer  (Appendix  A)  or  directed 
by  the  System  Program  Office.  At  this  time  the  negatives 
are  officially  transferred  to  the  prime  Air  Logistics 
Center  (ALC) , which  assumes  responsibility.  Figure  1, 

F-15  Technical  Data  Acquisition  Cycle,  displays  the 
sequence  of  events  in  the  cycle  (36). 

The  actual  acquisition  of  the  technical  data 
includes  considerable  coordination  in  the  form  of  con- 
tracts and  agreements.  The  process  is  initiated  in  the 
SPO  by  issuing  a Contract  Data  Requirement  List  (CDRL) 
which  defines  the  weapon  system  technical  data  require- 
ments. The  CDRL  consists  of  the  Technical  Order  Publi- 
cation Plan  H-101/M,  the  Technical  Manual  CFAE/CFE  notices 
H-105/M,  the  Aircraft  and  Training  Equipment  H107-1/M, 
and  the  Validation  Record  H-108/M.  These  elements  of  the 
contract  establish  delivery  conditions,  specification 
requirements,  and  validat  ion/verif  i.  cat  ion  (Appendix  A) 
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guidelines  for  the  entire  technical  data/order  acquisi- 
tion (36)  . 

Technical  Order  Costing  Method 

The  cost  of  providing  the  technical  order  (TO) 
negatives  to  the  USAP  on  contract  furnished  equipment  (CFE) 
and  TOs  that  require  inprocess  reviews  from  the  prime 
contractor  is  based  on  a Forward  Pricing  Agreement  (FPA) . 
This  agreement  between  the  USAF  and  the  contractor  speci- 
fies the  direct  cost  parameters  and  estimating  techniques 
that  are  used  to  determine,  as  applicable,  selected  cost 
elements  of  F-15  Change  Proposals  and  Notices  for  Publica- 
tions and  Data.  Basically  it  establishes  the  number  of 
manhours  required  per  page  of  a specific  category  of  TO. 

The  most  recent  FPA  for  the  F-15  divides  the  TO  pages  into 
major  categories  with  manhours  assigned  to  each  category 
or  subcategory.  This  categorization  is  depicted  in 
Table  1. 

The  FPA  takes  into  consideration  all  cases  and 
provides  an  average  manhour  figure  per  page  to  bill  the 
Air  Force.  The  following  hypothetical  example  briefly 
describes  the  costing  method  and  Figure  2 presents  a 
tabulation  summary  of  the  narrative  description. 

For  example,  a three  page  instruction  is  required 
to  correct  a deficiency  noted  in  maintenance  technical  data 
for  the  weapon  system.  This  Time  Compliance  Technical  Order 
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TABLE  1 

FORWARD  PRICING  AGREEMENT 


Category 

Manhours 

I. 

Airframe 

Fliqht  Manual 

5.0 

Mechanical  Maintenance 

5.3 

Electrical  Maintenance 

4.5 

Non-Nuclear 

10.0 

II. 

Contractor  Furnished  Equipment 
(CFE)  Manuals 

Ill . 

Contractor  Furnished  Air  Equipment 
(CFAE)  Manuals 

IV. 

Development  Program  Manuals 

Flight  Manuals 

4 . 0 

Inspection  Manuals 

Mechanical  Manuals 

2 . 6 

V. 

Retrofit  Data 

Time  Compliance  TO  (TCTO) 

Time  Compliance  Directive 

24  . o 

(Test  Aircraft) 

14.5 

VI. 

Inprocess  Review  Manuals 
(per  each  CFE  TO  only) 

Change 

7.0 

Initial 

59.0 

VII. 

Planning 

4.9 
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Pages  with  deficiencies  requiring 

correction  3 

TCTO  Title  Page  1 

List  of  Affected  Pages  1 

Total  pages  in  TCTO  5 

Manhours  per  page  x 24.6 

Manhours  per  TCTO  per  aircraft  model  123.0 

Models  of  aircraft  (F- 15A/B/C/D)  x 4 

Total  Manhours  492.0 

Manhours  for  planning  requirement  + 4 . 9 

496.9 

Test  aircraft  manhours  (14.5  x 5 pages)  + 72.5 

Grand  Total  Manhours  569.4 

569.4  times  labor  rate  per  manhour  equals  total  cost  of 
the  proposed  TCTO. 


Fig.  2. 


Summary  of  Cost  Computation  for  TCTO 
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(TCTO)  correction  would  fall  into  Category  V,  Retrofit 
Data;  and  five  separate  pages  would  be  required.  The  five 
pages  are:  the  TCTO  title;  a page  listing  the  affected 
pages  in  the  TO;  and  the  three  pages  of  corrected  main- 
tenance instructions.  Assuming  there  are  four  versions 
of  the  weapon  system  (F-15A,  B,  C,  and  D)  that  are  affected 
by  the  TCTO,  a separate  TCTO  would  be  published  for  each 
aircraft  at  a cost  of  24.6  manhours  per  page.  Therefore, 
the  five  pages  would  require  123  manhours.  Since  there 
are  four  versions  of  the  aircraft,  the  123  figure  multi- 
plied by  four  yields  492  manhours.  An  additional  4.9  man- 
hours must  be  added  for  planning  requirements  to  arrive  at 
a total  of  496.9  manhours  for  the  TCTO.  To  compute  the 
actual  total  cost  of  producing  the  TCTO,  the  496.9  man- 
hours is  multiplied  by  the  current  labor  rate  per  manhour. 
In  cases  where  special  test  aircraft  are  in  the  system, 
an  additional  14.5  manhours  per  page  are  added  to  accom- 
modate these  aircraft.  Thus,  72.5  manhours  (14.5  manhours 
times  5 pages)  are  added  to  496.9  prior  to  multiplying  tne 
cost  per  manhour  to  arrive  at  the  "grand  total"  cost  of 
researching,  modifying,  and  making  the  required  TCTO  nega- 
tives. 


Cost-Decision  Matrix 


The  F-15  SPO  is  faced  with  a complex  problem.  All 
new  technical  orders  are  required  to  be  procured  under  the 


procedural i zed  TO  format,  MIL-M-83495  (16).  Since  the 
F-15A  TOs  were  procured  under  the  old  format,  MIL-H-25098, 
the  U.S.  Air  Force  has  the  option  of  using  either  the  old 
or  new  format  for  future  procurements.  Therefore,  the  cost 
versus  benefits  of  the  proceduralized  TOs  must  be  analyzed 
to  determine  what  tradeoffs  exist. 

The  F-15  SPO  anticipated  the  requirement  for  cost 
data  for  the  various  alternatives  and  requested  Ball  Park 
cost  figures  from  McDonnell-Douglas  for  the  possible 
procurement  options.  Table  2 is  a cost-decision  matrix 
based  on  the  cost  data  provided  by  McDonnell-Douglas  <10:1). 

The  major  significance  of  the  cost-decision  matrix 
is  that  it  indicates  an  estimated  savings  of  $1.78  million 
can  be  achieved  by  procuring  the  F-15  A/B/C/D  aircraft 
TOs  as  a package  in  the  PTO  format.  This  savings  is  realized 
when  comparing  the  estimated  cost  of  procuring  the  PTOs 
in  separate  packages.  This  difference  in  cost  represents 
the  required  initial  cost  of  converting  the  available 
technical  data  of  the  F-15A  into  PTO  format.  After  the 
initial  conversion  has  been  accomplished,  the  future  cost  of 
procuring  the  PTO  would  be  approximately  the  same  as  the 
cost  of  procuring  additional  copies  under  the  CTO  format. 

The  printing  cost  of  PTOs  would  be  slightly  higher  because 
they  generally  contain  more  pages  than  a CTO. 
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TABLE  2 

COST-DECISION  MATRIX 


A/C  Package 

Cost  by 

TO  Format  Type* 

CTO 

PTO 

F-15A 

None 

Not  Available 

F-15B 

.2 

Not  Available 

F-15A/B 

.2 

4.8 

F-15C/D 

1.65 

5.28 

F-15A/B/C/D 

1.85 

8.3 

*In  millions  of  dollars  for  FY  1977  base  year. 
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There  is  a point  in  the  F-15  production  schedule 
where  the  option  to  procure  the  PTO  will  result  in  higher 
cost  because  it  will  not  be  possible  to  design  and  procure 
the  PTO  prior  to  the  actual  F-15  C/D  field  requirement. 
This  case  will  require  the  procurement  of  the  CTO  as  an 
interim  measure  while  the  PTO  is  being  developed.  This 
procurement  will  result  in  higher  LCC  cost,  because  total 
cost  must  include  not  only  the  additional  cost  of  the  CTOs 
but  also  the  opportunity  cost  incurred  by  sacrificing  any 
savings  that  might  have  been  achieved  due  to  the  inherent 
advantages  of  the  PTO. 


CHAPTER  V 


AN  EVALUATION  OF  PROCEDURAL! ZED  TOs  (PTOs) 

Introduction 

It  is  appropriate  at  this  time  to  restate  and  eval- 
uate the  postive  and  negative  factors  of  PTOs.  Each  factor 
will  be  evaluated  to  determine  if  it  is  feasible  to  quan- 
tify the  potential  cost  related  to  that  factor  using  the 
K051  data  base.  Those  factors  elected  for  further  analysis 
will  be  categorized  according  to  Work  Unit  Code  (WUC)  in 
order  to  extract  the  pertinent  data  from  the  K0S1  data 
base.  These  cost  data  will  be  subjected  to  Monte  Carlo 
simulation  to  determine  the  approximate  distribution  of 
the  total  cost  of  the  applicable  WUCs. 

Actually,  there  are  two  broad  categories  of  posi- 
tive and  negative  factors  for  job  guide  manuals  (JGMs) 
which  are  the  proceduralized  technical  orders  (PTOs)  being 
considered  for  the  F-15  weapon  system.  The  first  category. 
Factors  Affecting  Usability  and  Acceptance,  deals  with  how 
individual  maintenance  personnel,  the  users,  viewed  PTOs 
during  actual  "hands-on"  application  in  their  work  environ- 
ment. The  second  category,  Factors  Affecting  Maintenance 
Capability,  relates  to  how  PTOs  interact  with  the  entire 
weapon  system  maintenance  concept  to  change  mission  per- 
formance and  maintenance  costs. 
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Factors  Affecting  Usability  and  Acceptance 

The  1977  Air  Force  Human  Resources  Laboratory 
(AFHRL)  study  of  the  use  of  JGMs  and  logic  tree  trouble- 
shooti. g aids  (LTTAs)  produced  some  interesting  results. 

Iv  had  been  established  that  the  use  of  JGMs  and 
LTTAs  could  improve  maintenance  efficiency.  However, 
it  was  not  known  how  well  the  new  proceduralized  data 
would  be  accepted  by  the  users  or  what  problems  would 
be  encountered  in  implementing  and  using  the  data  in 
an  operational  environment  [15:1]. 

Positive  Factors.  The  following  positive  factors 
of  PTOs  were  noted  by  maintenance  personnel  who  participated 
in  the  study. 

1.  Size  of  Job  Guide  Manuals.  With  few  exceptions, 
technicians  agreed  that  the  smaller  size  made  the  JGMs 
easier  to  carry  and  use  (15:15). 

2.  Procedures  are  easy  to  read  and  understand.  The 
clear,  concise  manner  of  writing  in  the  JGMs  was  clearer  and 
often  more  thorough  than  CTOs  (15:15). 

3.  Presentations  of  Instructions . Technicians 
especially  liked  the  impact  conditions  page  which  tells  how 
to  prepare  for  the  job,  the  step-by-step  instructions  which 
tell  them  exactly  what  to  do,  the  illustrated  parts  break- 
down for  parts  information,  and  the  follow-on  maintenance 
instructions  (15:15). 

4.  Illustrations.  The  job  guide's  foldout  illus- 
trations, which  are  a key  part  of  the  maintenance  proce- 
dures, were  well  accepted  by  most  technicians  (15:15). 


5.  Dual-Level  Presentation.  Experienced  techni- 
cians greatly  appreciated  the  general  instructions  wnich 
enabled  them  to  complete  familiar  jobs  without  referring 

to  the  detailed  instructions  provided  for  the  inexperienced 
technicians.  This  dual-level  format  helped  reduce  expe- 
rienced personnel  resistance  to  the  change  to  JGMs  (15:16). 

6.  Format.  "Most  users  agreed  that  the  format 
makes  the  TO  easier  to  read,  understand,  and  use,  especially 
for  the  inexperienced  [15:25]." 

7.  Reading  Level.  "By  far,  the  predominant 
opinion  was  that  the  job  guides  are  very  easy  to  read,  to 
understand,  and  to  follow  [15:25]."  This  factor  was  most 
important  to  inexperienced  people,  but  even  the  experienced 
people  seemed  to  appreciate  the  ease  of  reading  and  under- 
standing (15:25). 

Negative  Factors.  C-141  aircraft  maintenance 
technicians  who  participated  in  the  study  also  detected 
negative  factors  of  the  PTOs.  The  following  list  is  a 
summary  of  the  negative  factors  affecting  usability  and 
acceptance . 

1.  Resistance  to  Change.  "There  is  a natural 
tendency  to  resist  anything  new  that  alters  one's  normal 
everyday  way  of  doing  things  [15:16]."  However,  this 
resistance  to  change  does  not  appear  to  be  significant  or 
long  lasting.  "A  surprising  number  of  those  who  expressed 
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dislike  or  indifference  to  the  data  indicated  that  the  prob- 
lem was  primarily  a matter  of  resistance  to  change  and  that 
they  will  probably  like  the  data  after  they  [are]  used  to 
it  [15:16]." 

2.  Errors  in  the  Data.  As  with  any  new  TO, 
errors  and  omissions  in  data  are  a serious  problem  in  gain- 
ing acceptance  of  the  data.  "However,  the  error  rate  did 
not  appear  to  be  any  greater  than  would  be  expected  for 
any  new  TO  [15:16]."  Because  of  the  nature  of  PTOs,  errors 
are  more  critical,  especially  when  the  PTO  is  used  by 
inexperienced  technicians.  The  prevailing  attitude  among 
most  technicians  was  that  they  will  have  to  live  with  the 
errors  until  they  can  be  corrected  through  the  AFTO  Form  22 
system  (15:16) . 

3.  Problems  in  Implementation.  There  were  four 
ma}or  implementation  problems  which  had  an  impact  on 
acceptance  of  the  PTOs. 

a.  A significant  communications  problem  was 
encountered  when  the  normal  chain  of  command  was  relied 
upon  as  the  major  means  of  getting  information  to  the  work- 
ing technicians.  A mass  briefing  on  the  purpose  of  the 
study  and  participation  required  from  the  technicians  would 
have  been  more  effective  (15:16). 

b.  "Adequate  instruction  on  the  use  of  the 
data  was  not  provided  [15:16]."  The  job  guides  themselves 
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were  not  difficult  to  use;  however,  since  the  technicians 


did  not  adequately  understand  what  is  contained  in  the  job 


guides  and  how  to  find  the  information,  their  effective- 


ness was  decreased  (15:16). 


c.  Policies  for  use  of  the  JGM  dual-level 


feature  were  not  adequately  explained  to  the  technicians, 


Again,  failure  to  communicate  effectively  resulted  in 


uncertainties  among  maintenance  personnel  concerning  MAC 


policy  on  use  of  the  dual-level  feature  (15:17) 


d.  "Adequate  provisions  were  not  made  for 


storing  the  JGMs  on  the  aircraft  [15:17]."  An  effective 


means  of  storing  the  manuals  on  the  aircraft  and  making 


them  readily  available  are  essential  to  encouraging  maxi- 


mum use  (15:17) . 


4.  Number  of  Volumes  Required.  "The  most  fre- 


quently heard  complaint  at  both  bases  [Charleston  and  Norton 


Air  Force  Bases]  was  that  too  many  books  are  required  to 


do  a job  [15:18]."  This  is  directly  contrary  to  the 


original  concept  that  all  information  required  to  do  a 


job  would  be  provided  in  one  job  guide  and  that  job  guides 


would  be  packaged  to  keep  the  number  of  volumes  required 


for  related  tasks  at  a minimum.  Air  Force  agencies  procur- 


ing JGMs  for  other  weapon  systems  should  give  special 


consideration  to  this  problem  (15:18). 
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5 . durability.  "A  problem  has  been  encountered 
with  the  pages  tearing  out  of  the  job  guide  manuals  [15:18]." 
The  problem  stems  from  three  factors:  poor  paper;  rough 
edges  on  binder  rings,  and  holes  punched  at  the  very  edge 
of  the  page.  All  of  these  deficiencies  should  be  corrected 
to  prevent  possible  serious  problems  caused  by  lost  pages 
(15:18)  . 

b.  illustration*.  Some  illustrations  were  found 
to  be  poor,  primarily  because  ol  the  printing  quality,  not 
because  of  the  content  (15:19). 

7.  bor.it  i n.j  I'rooedurc  s . Technicians  experienced 
serious  indexing  problems  with  the  new  technical  data  system 
because  some  data  contained  inadequate  index  and  table 

of  contents,  and  some  technicians  did  not  adequately 
understand  the  indexing  system.  Due  to  the  differences 
from  the  CTO  indexing  system,  training  on  locating  informa- 
tion should  be  an  integral  part  of  the  daily  implementation. 

8.  InoJ'fert  itu'  Proro.lurt’s  . "Technicians  complained 
that  some  procedures  were  ineffective  or  inefficient 
[15:19]."  The  two  most  common  complaints  were: 

a.  Experienced  technicians  complained  that 
checkout  procedures  are  sometimes  too  long.  The  quick  checks 
used  with  the  old  TOs  were  removed  and  replaced  with  new, 
longer  checkouts  in  some  cases. 


b.  The  technicians  said  that  the  new  procedures 
require  the  use  of  far  more  danger  tags  than  the  CTOs. 

"Since  danger  tags  are  a controlled  item,  any  unnecessary 
use  of  the  tags  presents  an  unnecessary  administrative 
requirement  and  a substantial  increase  in  maintenance  time 
(15:19) 

9.  Omitted  Procedures.  Although  the  new  data 
cover  many  more  tasks  than  the  CTOs,  technicians  report 
that  some  tasks  have  been  omitted.  It  is  likely  that  some 
procedures  were  in  fact  omitted  while  others  simply  were 
not  found  by  the  technicians  (15:19). 

10.  Manpower  acquirements.  "Procedures  sometimes 
require  more  men  than  are  normally  used  or  available  [15:19) 

11.  Incomplete  Troubleshooting  Data.  "Many 
common  malfunctions  were  not  included  in  the  malfunction 
indexes  or  covered  by  troubleshooting  trees  (15:20)."  Again 
the  verification  and  validation  phases  must  be  thorough 
enough  to  insure  that  the  troubleshooting  data  are  fully 
capable  of  serving  the  intended  purpose. 

Summary . Obviously,  both  the  positive  and  negative 
factors  affecting  usability  and  acceptance  of  the  TTOs  are 
often  highly  subjective  in  nature  because  they  relate  to 
individual  user  preferences.  Consequently,  any  attempts  to 
quantify  and  assign  dollar  values  to  these  factors  would  be 
futile.  Certainly,  these  factors  in  some  way  affect  main- 


~ ! — ^srr  ■ ttw 


il-  . 


tenance  costs;  unfortunately,  there  is  no  practical 
method  of  measuring  their  effect.  In  the  long  range  the 
overall  effect  should  result  in  cost  savings?  but:,  for  the 
purpose  of  this  thesis,  it  is  assumed  that  the  positive 
and  negative  factors  will  at  worst  negate  each  other. 

Most  of  the  shop  supervisors  interviewed  spoke 
well  of  the  concept  and  the  potential  of  the  job 
guides.  They  recognized  the  problems,  especially 
the  errors  and  sometimes  lengthy  checkout  procedures, 
but  overall  they  indicated  that  the  job  guides  were 
much  better  than  the  original  TOs.  The  officers  and 
senior  NCOs,  almost  without  exception,  were  optimistic 
about  the  long-range  impact  of  the  job  guides  upon 
C-141  aircraft  maintenance  [15:28]. 


Factors  Affecting  Maintenance  Capabi 1 i ty 
The  primary  source  of  factors  affecting  actual 
maintenance  capability  and  performance  is  the  197b  Air 
Force  Human  Resources  Laboratory  (AF11RL)  Evaluation  of 
Three  Types  of  Technical  Data  for  Troubleshooting.  The 
results  of  this  experimental  evaluation  clearly  demonstrate 
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that  the  use  of  PTOs  resulted  in  significantly  better 
troubleshooting  than  the  use  of  CTOs  (15:2). 

Positive  Factors.  The  analysis  of  the  data 
collected  in  the  AFHRL  study  indicated  that,  given  tech- 
nically accurate  PTOs  of  high  quality,  the  following 
positive  factors  can  be  identified. 

1.  Apprentice  electronics  technicians  with  no 
field  experience  are  able  to  troubleshoot  moderately 
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complex  electronic  systems  effectively  (19:90). 

2.  Technicians  with  more  than  six  months  experience 
on  moderately  complex  electronic  systems  are  able  to 
troubleshoot  more  effectively  with  PTOs  than  the  standard 
CTOs  at  the  intermediate  level  of  maintenance  (19:90). 

3.  Apprentice  technicians  with  no  field  experience 
using  PTOs  are  able  to  troubleshoot  as  effectively  or  more 
effectively  at  the  intermediate  level  than  technicians 
with  some  experience  (less  than  six  months)  using  standard 
CTOs  (19:90). 

4.  The  use  of  PTOs  results  in  significantly  fewer 
"good"  parts  being  replaced  unnecessarily  than  when  CTOs 
are  used.  Use  of  PTOs  at  the  intermediate  maintenance 
level  in  an  operational  environment  could  result  in  signi- 
ficant cost  savings  due  to  reduced  spare  parts  consumed 
(19:90)  . 

5.  Even  for  experienced  technicians,  trouble- 
shooting at  the  intermediate  level  is  performed  signifi- 
cantly faster  when  PTOs  are  used  (19:90). 

6.  Experienced  technicians  believe  that  the  PTOs 
are  less  difficult  to  understand  than  the  CTOs  (19:91). 

7.  Experienced  technicians  prefer  the  PTOs  over 
the  CTOs  for  use  in  troubleshooting  at  all  levels  oi 
complexity  from  flightline  to  intermediate  maintenance  to 
depot  repair  (19:91). 


8.  "Although  the  present  study  was  limited  to 
electronic  maintenance,  it  is  highly  probable  that  similar 
results  would  be  obtained  in  other  maintenance  areas 
[19:911 . " 

9.  A greater  proportion  of  the  maintenance 
problems  were  solved  using  the  PTOs  (19:66). 

10.  In  addition  to  the  proven  advantages  of  PTOs 
documented  during  actual  field  testing,  the  Air  Force 
Logistics  Management  Center  reported  some  additional 
positive  aspects  of  PTOs  in  their  Initial  Technical  Order 
Project  Findings.  These  positive  factors  are  briefly 
summarized  below. 

a.  PTOs  can  reduce  long  range  maintenance 
cost  (29:23) . 

b.  PTOs  simplify  training  and  maintenance 
actions  (29 : 23) . 

c.  The  new  TO  system  uses  a numbering  system 
that  is  in  use  internationally  (29:23). 

d.  The  gap  between  designers,  engineers,  and 
maintenance  personnel  is  bridged  by  PTOs  (29:23). 

e.  PTOs  establish  better  configuration  control 
of  aircraft  wiring  (29:23). 

f.  Time  spent  in  search  and  retrieval  of 
technical  data  is  reduced  by  the  proceduralized  system 
(29:24) . 
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counterproductive  than  the  problems  currently  being  experi- 
enced with  CTOs. 
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Factors  Selected  for  Further  Analysis 
Positive  Factors.  The  AFHRL  field  study  which  com- 
pared three  types  of  technical  data  for  troubleshooting  gen- 
erated very  quantitative  and  rigorous  statistics  relevant 
to  number  of  problems  solved,  performance  of  apprentice  and 
experienced  technicians,  number  of  spare  parts  consumed,  and 
time  required  to  troubleshoot  when  both  PTOs  and  CTOs  were 
used  for  electronic  maintenance  troubleshooting.  These  pos- 
itive factors  are  used  in  the  following  chapter  as  the  basis 
for  computing  the  cost  savings  which  might  be  achieved  using 
the  PTOs  instead  of  the  CTOs  currently  in  use. 

The  Analogous  Systems  Approach 
Method . The  AFHRL  study  was  limited  to  two  elec- 
tronic subsystems  of  the  C-141  aircraft  as  a foundation  for 
comparison  tests  of  three  types  of  technical  data  for 
troubleshooting.  The  subsystems  selected  were  the 
AN/APN-147  and  the  AN/ASN-35  which  are  both  parts  of  the 
doppler  radar.  Consequently,  a strong  analogy  to  the  F-15 
avionics  subsystems  can  be  established.  If  the  advantages 
of  PTOs  are  valid  for  the  test  subsystems  (the  AN/APN-147 
and  the  AN/ASN-35) , then  the  magnitude  of  these  advantages 
is  expected  to  be  at  least  as  great  or  even  greater  for  the 
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more  complex  F-15  avionics  systems.  Therefore,  the  follow- 
ing seven  work  unit  codes  (WUCs)  were  selected  to  develop 
the  F-15  potential  cost  savings:  51XXX-Instruments; 
52XXX-Autopilot;  57XXX-Integrated  Guidance  and  Flight  Con- 
trol System,.  63XXX-IFF  System;  7lXXX-Radio  Navigation; 
and  74XXX-Fire  Control  System.  The  K051  file  data  extracted 
for  these  WUCs  represents  the  cost  of  maintaining  the 
F-15  avionics  subsystems. 

The  analogy  between  C-141  and  F-15  avionics 
maintenance  is  useful  in  developing  cost  data  for  two 
reasons.  First,  avionics  electronic  subsystems  constitute 
the  highest  maintenance  cost  for  the  F-15  weapon  system. 
Second,  if  cost  savings  using  PTOs  can  be  documented  for 
the  C-141  aircraft,  it  is  reasonable  to  assume  that  those 
cost  savings  would  also  be  valid  for  the  more  complex 
F-15  avionics  subsystems. 

Data  Limitations.  There  are  three  primary  limita- 
tions to  the  data  gathered  to  draw  the  analogy  between  the 
C-141  and  F-15  systems.  First,  there  are  many  positive 
aspects  of  the  PTOs  that  cannot  be  quantified  in  terms  of 
dollars  saved.  Although  there  is  no  way  to  estimate  these 
savings,  they  are  expected  to  act  as  hidden  multipliers 
and  create  significant  savings  in  maintenance  dollars  (29:23). 
Second,  the  AFHRL  study  compared  PTOs  and  CTOs  only  on  a 
limited  application  to  two  avionics  subsystems.  The 
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effect  of  job  guide  manuals  on  the  entire  C-141  aircraft 
will  not  be  known  until  more  data  on  maintenance  costs 
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after  the  implementation  of  PTOs  can  be  gathered  and 
analyzed.  Finally,  critics  of  the  K051  data  system 
point  out  that  the  figures  are  in  error  because  they 
consistently  underestimate  the  actual  cost  of  each  WUC. 
However,  if  a cost  savings  is  realized  despite  the  con- 
servative (low)  K051  cost  data  values,  then  the  actual 
savings  should  be  even  greater  than  estimated  by  this 
thesis . 
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Introduction 
■ 

The  Data  Analysis  chapter  is  divided  into  three 

' 

mam  parts.  The  first.  Performance  Comparison  for  CTOs 


and  PTOs,  deals  with  estimating  the  potential  percentage 
savings  which  can  be  achieved  utilizing  PTOs  instead  of 
CTOs.  The  second  part,  Forecast  of  F-15  Logistics  Cost, 
uses  three  years  of  F-15  maintenance  cost  data  extracted 
from  the  IROS  K051  data  file  to  predict  the  future  cost  of 
avionics  maintenance.  This  forecast  represents  the  pro- 
jected cost  of  using  CTOs;  thus, the  estimated  dollar  sav- 
ings for  PTOs  can  be  computed  by  applying  the  percentage 
savings  factors  obtained  in  the  first  part  of  the  data 
analysis.  The  third  part  contains  an  analysis  of  three 
break-even  charts.  These  charts  display  the  predicted 
break-even  points  (the  points  in  time  where  savings  gener- 
ated by  PTOs  equal  the  increased  cost  of  acquiring  the 
PTOs)  for  the  anticipated  savings  derived  from  the  improved 
technical  data. 
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Performance  Comparison  for  CTOs  and  PTOs 
Measuring  Performance 

As  defined  by  Robert  N.  Anthony,  performance  con- 
sists of  two  dimensions  (3:377).  The  first,  effectiveness, 
expresses  how  well  a maintenance  technician  accomplishes 
che  job  measured  in  terms  of  planned  and  actual  output. 

The  ratio  of  actual  jobs  accomplished  to  total  jobs 
assigned  yields  the  technicians  percentage  of  effective- 
ness. The  second  dimension  of  performance,  efficiency, 
deals  with  the  consumption  of  input  resources  to  accom- 
plish the  job.  Therefore,  efficiency  is  stated  as  the 
ratio  of  inputs  (parts  or  time)  consumed  to  outputs  (jobs) 
achieved. 

The  evaluation  of  three  types  of  technical  data 
for  troubleshooting  conducted  by  the  AFHRL  tested  three 
performance  measures  of  maintenance  personnel.  One  effec- 
tiveness measure  and  two  efficiency  measures  of  the  per- 
formance of  these  technicians  were  recorded  during  the 
experiment.  Mean  proportion  of  problems  solved  is  an 
effectiveness  measure,  while  mean  number  of  parts  used 
incorrectly  and  mean  time  to  troubleshoot  and  repair  are 
efficiency  measures.  The  combination  of  these  three 
measures  can  be  used  to  estimate  the  improvement  in  over- 
all performance  as  a result  of  using  proceduralized  tech- 
nical data. 
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Results  of  the  AFHRL  Technical 
Data  Evaluation 

The  effectiveness  of  technical  data  was  evaluated 
by  measuring  the  ability  of  technicians  to  troubleshoot 
representative  faults  in  the  test  systems  using  each  type 
of  data.  Job  performance  was  used  to  indicate  trouble- 
shooting ability.  Each  test  was  administered  by  inserting 
a bad  component  into  the  system  and  observing  the  perform- 
ance of  the  technician  as  he  used  one  of  the  three  types 
of  data  to  identify  the  bad  component.  Data  was  recorded 
regarding  whether  the  fault  was  correctly  isolated,  the 
number  of  spare  parts  used,  and  the  time  required  to 
troubleshoot  (20:9). 

Data  were  collected  from  personnel  in  three  cate- 
gories: recent  technical  school  graduates  (no  field  exper- 

ience), technicians  with  six  months  or  less  experience  on 
the  equipment,  and  technicians  with  more  than  six  months 
experience.  An  analysis  was  conducted  on  each  category 
of  technicians  separately,  and  then  the  combined  results 
were  studied  to  determine  the  total  improvement  in  mainte- 
nance performance  using  PTOs . 


Apprentice  Technicians.  Eighteen  airmen  who  had 
just  graduated  from  Keesler  Technical  Training  Center 
(KTTC)  Course  No.  3ABR32834  served  as  subjects  for  the 


study  (20:9).  They  were  randomly  assigned  to  two  groups; 


one  group  used  FPTAs  first  and  the  other  used  PTOs  first. 
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The  apprentice  technicians  performed  one-half  of  the  prob- 
lems with  FPTAs  and  one-half  with  PTOs.  A controlled 
sequence  of  problem  assignment  ensured  that  each  problem 
was  performed  an  equal  number  of  times  with  each  type  tech- 
nical data.  This  sequence  also  ensured  that  each  problem 
was  controlled  for  possible  order  effects  in  the  experi- 
ment. These  technicians  were  not  tested  using  CTOs  since 
this  level  technician  is  not  expected  to  be  able  to  trouble- 
shoot effectively  using  the  CTOs  (18:9). 

Despite  this  apparent  lack  of  capability  with  CTOs, 
the  apprentice  technicians  performed  extremely  well  with 
PTOs.  As  indicated  in  Table  3,  the  effectiveness  and 
efficiency  of  apprentice  technicians  using  PTOs  approached 
the  performance  measures  for  experienced  technicians 
using  CTOs.  This  initial  capability  to  complete  the  job 
without  on-the-job  training  should  have  a great  effect 
on  personnel  and  training  requirements.  Consequently, 
the  performance  capability  of  the  apprentice  technicians 
would  probably  result  in  significant  savings. 


Technicians  with  Six  Months  or  Less  Experience. 
Thirty-six  technicians  in  the  Military  Airlift  Command 
served  as  subjects  in  the  two  experienced  categories. 
Eighteen  had  six  months  or  less  experience  on  the  equip- 
ment and  eighteen  had  more  than  six  months  experience 
(18:9).  The  subjects  included  both  military  and  civilian 


SUMMARY  OF  THE  AFHRL  EXPERIMENT  RESULTS 


1 


personnel,  and  the  same  experimental  procedures  were  used 
for  both  groups.  A problem  assignment  procedure  similar 
to  that  used  for  the  apprentice  subjects  was  used;  how- 
ever, the  experienced  technicians  were  tested  with  all 
three  types  of  technical  data  (18;9). 

The  effectiveness  (mean  proportion  of  problems 
solved)  for  the  technicians  with  six  months  or  less 
experience  at  the  organizational  level  of  maintenance 
was  100  percent  for  both  CTOs  and  PTOs . Their  efficiency 
(mean  time  to  troubleshoot  and  repair)  was  actually  better 
using  CTOs;  however,  a large  number  of  parts  were  used 
incorrectly.  Parts  used  incorrectly  can  become  an 
extremely  costly  item  because  of  the  total  logistics 
cost  required  to  procure,  handle,  transport,  and  maintain 
these  items.  Thus,  significant  savings  in  fewer  parts 
used  incorrectly  should  easily  offset  an  average  of  12.4 
minutes  additional  time  consumed  in  troubleshooting  using 
PTOs . 

At  the  intermediate  maintenance  level  the  per- 
formance using  PTOs  was  superior  to  all  three  measurement 
criteria.  Perhaps  the  most  impressive  difference  in  per- 
formance was  the  mean  number  of  parts  used  incorrectly. 

The  average  technician  used  over  2.5  times  as  many  parts 
incorrectly  when  troubleshooting  with  CTOs.  In  addition, 
technicians  using  PTOs  solved  28.8  percent  more  problems 
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while  using  an  average  of  25.1  minutes  less  time  per 
solution. 
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Technicians  cith  More  Than  Six  Months  Experience. 
Eighteen  of  the  thirty-six  technicians  in  the  328X4  AFSC 
had  more  than  six  months  experience,  and  these  subjects 
were  tested  in  the  same  manner  as  the  first  group  of 
experienced  technicians.  It  was  noted  that  these  more 
experienced  technicians  solved  more  problems  in  less  time 
using  CTOs  at  the  organizational  level  of  maintenance. 
However,  they  again  used  almost  three  times  as  n ny  parts 
incorrectly  when  troubleshooting  with  CTOs. 

At  the  intermediate  level,  perfornuui  ..  . tng  the 
PTOs  was  as  good  or  better  than  the  portorman.  , »i  CTOs 

in  all  three  performance  measurements. 

Summary  of  Potential  Savings 

Even  without  the  possible  savings  due  tv)  having 
apprentice  technicians  accomplish  more  tasks,  it  is  rea- 
sonable to  expect  a considerable  reduction  in  the  F-16 
avionics  maintenance  costs  as  a result  of  improved  per- 
formance by  experienced  technicians  troubleshooting  with 
PTOs.  Table  4 presents  a summary  of  the  percentage  of 
savings  for  each  performance  measure  evaluated  during  the 
AFHRL  testing  of  exprienced  avionics  maintenance  personnel. 
All  figures  in  Table  4 are  expressed  in  percentages. 
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TABLE  4 


SUMMARY  OF  ESTIMATED  PERCENTAGE  SAVINGS  FOR  PTOs 


Percentage 

Savings 


0 

34.0 
-77.5  * 


28.8 

60.7 

41.9 


- 4.2  * 
66.0 
-44.5  * 


9.0 

0 

15.6 


*Percentage  savings  in  favor  of  CTOs. 
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Six  Months  or  Less  Experience 


Organizational  Maintenance 

Mean  proportion  of  problems  solved 
Mean  number  or  parts  incorrectly  used 
Mean  time  to  troubleshoot  and  repair 

Intermediate  Maintenance 

Mean  proportion  of  problems  solved 
Mean  number  of  parts  incorrectly  used 
Mean  time  to  troubleshoot  and  repair 

More  than  Six  Months  Experience 

Organizational  Maintenance 

Mean  proportion  of  problems  solved 
Mean  number  of  parts  incorrectly  used 
Mean  time  to  troubleshoot  and  repair 

Intermediate  Maintenance 

Mean  proportion  of  problems  solved 
Mean  number  of  parts  incorrectly  used 
Mean  time  to  troubleshoot 


The  original  data  for  mean  proportion  of  problems 
solved  were  already  expressed  in  percentages;  therefore, 
it  was  possible  to  make  a direct  comparison  of  the  effec- 
tiveness of  CTOs  and  PTOs.  The  numbers  opposite  the  Mean 
Proportion  of  Problems  Solved  in  Table  4 represent  the 
percentage  difference  in  effectiveness  between  the  two 
types  of  technical  data. 

The  experimental  data  for  mean  number  of  parts 
incorrectly  used  and  for  mean  time  to  troubleshoot  and 
repair  were  not  expressed  as  percentages.  By  using  the 
CTO  efficiency  as  a standard  for  comparison,  the  data 
were  converted  to  percentages  which  reflect  the  improve- 
ment in  efficiency  derived  from  using  PTOs.  The  nega- 
tive figures  shown  in  Table  4 indicate  that  the  potential 
percentage  savings  for  that  specific  performance  measure 
actually  favors  the  CTOs. 

Although  the  elements  of  performance,  effective- 
ness and  efficiency,  could  not  realistically  be  combined 
to  compute  an  overall  average,  visual  inspection  of 
Table  4 indicated  that  a significant  savings  could  be 
anticipated  with  PTOs.  The  best  approach  to  the  problem 
was  to  estimate  a realistic  range  of  savings  that  might 
be  expected.  In  order  to  display  a greater  range  of 
possible  savings  estimates,  the  following  savings  options 
were  selected:  "pessimistic"  = 3 percent;  "most  likely" 

= 15  percent;  and  "optimistic"  = 27  percent.  These 
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figures  are  consistent  with  the  savings  expectations 
expressed  by  PIMO^  and  other  studies  covered  in  the 
Literature  Review.  Research  psychologists  from  the  AFHRL, 
who  were  directly  involved  in  the  technical  data  evalua- 
tion project,  also  supported  these  figures  as  reasonable 
estimates  of  expected  cost  savings  (5:25). 

Forecast  of  F-15  Logistics  Cost 

The  available  data  for  the  seven  applicable  WUCs 
(51XXX — Instruments,  52XXX — Autopilots,  5 7 XXX — Integrated 
Guidance  and  Flight  Control  Systems,  6 3 XXX — IFF  Systems, 
71XXX — Radio  Navigation,  and  74XXX--Fire  Control  System) 
were  collected  from  the  K051  (see  IROS  in  Chapter  II) 
data  base.  The  data  were  expressed  in  average  monthly 
cost  by  WUC,  by  quarter;  therefore,  simple  mathematical 
calculations  were  required  to  express  the  data  in  average 
cost  per  aircraft  per  quarter  (see  Table  5) . 

For  each  WUC,  cost  per  aircraft,  was  plotted 
over  a time  horizon.  These  graphs,  found  in  Appendix  C, 
were  used  to  visually  display  the  general  trend  of  each 
individual  WUC.  Simple  linear  regression  was  used  to 
find  a mathematical  trend  for  future  cost.  The  visual 
displays  and  mathematical  trends  were  compared  to  the 


^"Compared  to  the  PIMO  estimates,  15  percent  is 
a rather  conservative  estimate  of  potential  savings. 
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SIMFIT  (see  SIMFIT  in  Chapter  II  and  Appendix  C)  com- 
puter program  results.  The  Kolmogorov-Smirnov  (K-S)  test 
was  used  over  the  Chi-square  test  because  it  is  the 
more  appropriate  and  powerful  test  when  the  data  base  is 
relatively  small  (32:14).  Additionally,  as  indicated 
in  Table  6,  more  than  one  distribution  was  acceptable 
at  the  90  percent  confidence  level.  This  was  not  unex- 
pected as  the 

. . . Erlang,  Lognormal,  Gamma,  and  Weibull  dis- 
tributions are  basic  in  the  theory  of  curve  fitting 
and  when  the  sample  is  small,  it  is  difficult  to 
clearly  distinguish  between  the  various  distri- 
butions. For  example,  all  four  of  the  distributions 
may  fit  the  data  and,  therefore,  the  researcher  has 
to  decide  which  probability  distribution  best  repre- 
sents his  data  [32:6]. 

Various  combinations  of  acceptable  variable  dis- 
tributions were  tested  to  determine  their  sensitivity  to 
change.  It  was  found  that  the  mean  values  and  the  total 
probability  distributions  changed  insignificantly  when 
the  individual  WUC  distributions  were  varied.  The  distri- 
butions that  were  determined  to  best  describe  the  indi- 
vidual variables  (WUCs)  were  then  used  in  the  Monte  Carlo 
(see  Monte  Carlo  in  Chapter  II  and  Appendix  D)  computer 
simulation  program.  A total  of  4,000  convolutions  of 
the  random  variable  were  generated  to  form  the  Avionics 
Maintenance  Cost  Probability  Distribution  in  Figure  3. 
Three  points  on  the  total  cost  probability  distribution 
(low,  middle,  and  high)  were  selected  to  answer  the  thesis 
research  question.  The  high  point  represents  the  point 
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where  there  is  only  a 3 percent  probability  that  the 
future  cost  would  be  greater  than  $67,727.  In  other  words, 
there  is  a 97  percent  chance  that  the  future  cost  would  be 

lower  than  $67,727.  The  low  point  indicates  that  there 

is  a 3 percent  chance  that  the  future  cost  will  be  lower 
than  $5,513  and  a 97  percent  chance  that  it  will  be  higher 
than  $5,513.  The  middle  cost  is  the  point  on  the  proba- 
bility distribution  where  there  is  a 50  percent  chance 
that  the  future  cost  will  be  higher  than  $17,211  and  a 
50  percent  chance  that  it  will  be  lower  than  $17,211. 

All  of  the  above  figures  are  for  FY  1974;  there- 
fore, these  figures  were  adjusted  to  FY  1977  base  year 

using  the  National  Defense  Budget  Estimates  for  t'Y  1977. 
This  publication  provides  the  DOD/MAP  composite  index 
which  is  weighted  to  include  the  ratio  of  civil  service 
pay,  military  personnel  pay,  retired  pay,  and  industry 
purchases  for  all  DOD  operations  (33:78).  The  composite 
FY  1974  index  was  77.2,  and  the  adjusted  forecast  cost 
figures  are  (low — $7,141,  middle — $22,294,  and  high — 
$87,729)  per  aircraft  per  quarter. 

Table  7 is  a matrix  summary  of  the  PTO  percentage 
savings  options  (pessimistic,  most  likely,  and  optimistic) 
and  the  forecast  F-15  logistics  (low,  middle,  and  high) 
cost  probabilities.  The  combination  of  three  options 
and  three  probabilities  for  each  option  yields  nine  savings 
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forecasts  which  range  from  $214  to  $23,687  per  aircraft 
per  quarter  for  the  seven  avionics  WUCs . 


Break-Even  Analysis 

Three  break-even  charts  were  developed  from  the 
potential  PTO  savings  shown  in  Table  7.  Since  these 
figures  represent  dollars  per  aircraft,  it  was  necessary 
to  obtain  a production  schedule  for  the  remainder  of  the 
production  cycle.  The  F-15  Production  Schedule  in  shown 
in  Table  8 (4).  With  this  information,  the  savings  per 
quarter  were  computed  by  multiplying  the  theoretical 
savings  per  aircraft  by  the  total  number  of  aircraft  in 
the  inventory  during  each  quarter.  Those  aircraft  in  pro- 
duction during  any  quarter  were  not  added  to  the 
total  until  the  following  quarter. 

One  break-even  chart  was  developed  for  each  FTO 
savings  option.  For  example,  the  Pessimistic  Break-Even 
Chart  in  Figure  4 is  based  on  3 percent  savings.  The 
three  savings  curves  on  this  chart  reflect  the  effect  of 
3 percent  savings  at  each  of  three  points  on  the  proba- 
bility distribution  for  F-15  forecast  avionics  maintenance 
cost.  A low,  middle,  and  high  cost  figure  was  selected 
from  the  distribution  as  shown  in  Figure  3.  The  Most 
Likely  and  Optimistic  chart  were  drawn  in  the  same 
manner.  The  following  paragraphs  provide  a brief  descrip- 
tion and  analysis  of  each  break-even  chart. 


79 


r 


h ■ ■ - 


n 


i 


TABLE  8 

F-15  PRODUCTION  SCHEDULE 


Fiscal 

Year 

Quarter 

Aircraft  Scheduled 
for  Production 

Total  Aircraft  at 
Beginning  of  the 
Quarter 

79 

1 

27 

334 

2 

27 

361 

3 

27 

388 

4 

24 

415 

80 

1 

24 

439 

2 

24 

463 

3 

23 

487 

4 

17 

510 

81 

1 

21 

527 

2 

21 

548 

3 

17 

569 

4 

19 

586 

82 

1 

20 

605 

2 

20 

625 

3 

19 

644 

4 

18 

663 

83 

1 

18 

681 

2 

18 

699 

3 

12 

717 

4 

• • • 

729 
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Pessimistic  Bre>ak-Even  Chart 


Referring  to  Table  7,  the  smallest  savings  con- 
dition (3  percent  savings  and  the  low  avionics  maintenance 
cost)  yields  a savings  oi  $ 214  per  aircraft  per  quarter. 
Even  under  this  minimum  savings  condition  the  break-even 
point  would  be  achieved  during  the  second  quarter  of  FY  90 
which  is  well  within  the  lifetime  of  the  F-15  aircraft. 
This  information  is  displayed  graphically  in  Figure  4. 

The  middle  probability  (50  percent)  savings  curve 
indicates  that  the  break-even  point  would  be  achieved 
during  the  second  quarter  of  FY  83.  The  final  curve  on 
this  chart  depicts  the  cumulative  expected  savings  at  a 
rate  of  $2,632  per  aircraft  per  quarter.  This  number  is 
based  on  a 3 percent  savings  where  the  F-15  avionics 
maintenance  costs  are  unusually  high.  In  this  case  the 
break-even  point  would  be  achieved  within  twenty-one 
months  after  implementing  the  PTOs. 

Most  Likely  Break-Even  Chart 

The  Most  Likely  Break-Even  Chart  shown  in  Figure  5 
is  perhaps  the  most  important  of  the  three  charts.  It 
reflects  the  result  of  a potential  savings  of  15  percent 
upon  the  F-15  avionics  maintenance  cost  figures  selected 
from  the  cost  probability  distribution. 

Where  the  future  cost  is  expected  to  be  low,  the 
potential  savings  is  $1,071  per  aircraft  per  quarter.  It 
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the  F-15  production  plan  runs  on  schedule,  the  break-even 
point  would  be  reached  during  the  first  quarter  of  FY  82. 
The  most  significant  savings  curve  on  any  of  the  charts 
depicts  the  effect  of  a most  likely  savings  of  15  percent 
upon  the  most  probable  maintenance  cost  per  aircraft  per 
quarter.  This  curve  indicates  that  a break-even  point 
would  be  achieved  during  the  first  quarter  of  FY  80.  With 
all  factors  taken  into  consideration,  FY  80  is  perhaps  the 
most  realistic  forecasted  break-even  point.  The  last 
savings  curve  on  this  chart  was  plotted  based  on  a high 
savings  of  $13,159  per  quarter,  and  the  break-even  inter- 
cept would  occur  during  the  second  quarter  of  FY  79. 

Optimistic  Break-Even  Chart 

The  final  break-even  chart  appears  in  Figure  6. 
This  chart  represents  the  most  optimistic  savings  antici- 
pated which  is  27  percent  of  the  F-15  avionics  maintenance 
cost.  The  most  conservative  savings  curve  on  this  chare 
is  based  upon  a low  savings  of  $1,928  per  aircraft  per 
quarter.  In  this  case  the  break-even  situation  would 
be  achieved  during  the  third  quarter  of  FY  80.  The  mid- 
dle savings  of  $6,019  was  used  to  plot  the  next  curve,  and 
the  break-even  position  would  be  attained  during  the  third 
quarter  of  FY  79.  Finally,  the  last,  savings  curve  was 
based  upon  the  situation  where  the  avionics  maintenance 
cost  was  extremely  high  and  a 27  percent  savings  was 
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achieved.  In  this  very  optimistic  savings  case,  break- 
even would  be  realized  during  the  first  quarter  of  FY  79. 
In  contrast,  in  Figure  4 the  break-even  point  under  the 
most  pessimistic  conditions  was  midway  into  FY  90. 

Summary 

This  chapter  presented  a three-part  data  analysis. 
The  results  of  the  AFHRL  experimental  test  of  three  types 
of  technical  data  for  troubleshooting  were  evaluated  in 
order  to  estimate  the  potential  percentage  savings  which 
can  be  achieved  from  the  PTO  format.  Three  savings 
options  were  selected;  pessimistic-- 3 percent;  most  likely 
--15  percent;  and  optimistic — 27  percent.  Next,  F-15 
maintenance  cost  data  for  seven  avionics  WUCs  over  three 
years  were  extracted  from  the  K015  data  file.  Visual 
inspection,  linear  regression,  and  SIMFIT  analysis  were 
used  to  determine  the  most  appropriate  distribution  for 
each  WUC.  These  data  were  then  used  to  develop  a Monte 
Carlo  simulation  of  the  total  forecasted  future  cost  of 
the  seven  WUCs.  Three  cost  figures  were  picked  from  the 
resulting  distribution.  There  is  a 97  percent  probability 
that  the  actual  future  cost  will  be  greater  than  the  low 
cost  value,  and  there  is  a 97  percent  probability  that 
the  actual  future  cost  will  be  less  than  the  high  figure. 
The  middle  figure  on  the  cost  distribution  is  the  point 
where  there  exists  a 50  percent  probability  that  the 
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future  cost  will  be  higher  and  a 50  percent  chance  that 
the  future  cost  will  be  lower. 


All  dollar  values  were  adjusted  to  FY  77,  and  a 
three-by- three  matrix  was  constructed  to  visually  display 
the  data.  Finally,  the  break-even  analysis  charts  illus- 
trate the  forecast  points  in  future  time  where  the  savings 
for  each  of  the  nine  states  of  nature  equals  the  additional 
cost  of  procuring  the  PTOs.  It  was  determined  that  the 
most  likely  break-even  point  would  be  achieved  during  the 
first  quarter  of  FY  80. 
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CHAPTER  VII 

SUMMARY,  CONCLUSIONS,  AND 
RECOMMENDATIONS 

Summary 

For  many  years,  the  United  States  Air  Force  (USAF) 
has  expressed  strong  interest  in  finding  methods  of 
reducing  weapon  system  maintenance  cost.  "One  way  of 
reducing  cost  is  to  increase  the  productivity  of  mainte- 
nance personnel  by  providing  better  technical  data  [18:1]." 
The  results  of  "A  Study  of  the  Air  Force  Maintenance 
Technical  Data  System,"  accomplished  by  the  Behavioral 
Sciences  Laboratory  in  1962,  identified  the  need  for  an 
improved  technical  order  (TO)  format  in  the  USAF  (2:44). 
High  maintenance  personnel  cost  makes  it  imperative  that 
a more  efficient,  improved,  proceduralized  TO  (PTO)  for- 
mat, such  as  the  logic  tree  troubleshooting  aid  ( LTTA) , 
be  developed  and  utilized. 

At  the  present  time,  LTTAs  are  being  procured  for 
the  F- 16  aircraft  and  all  other  new  aircraft  acquisitions 
as  part  of  the  original  technical  data  package  (22) . Due 
to  the  significant  advantages  of  PTOs,  the  F-15  Technical 
Order  Management  Agency  (TOMA)  manager  is  in  the  process 
of  evaluating  the  cost  of  converting  the  TOs  for  the  F-15 
to  the  PTO  format.  The  F-15A  model  has  already  been 
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procured  with  the  conventional  TOs  jCTOs) , but  there  is  now 
an  opportunity  to  purchase  the  new  PTOs  for  various  combin- 
ations of  the  F-15A/B/C/D  models.  A cost  versus  benefits 
analysis  of  the  advantages  and  disadvantages  of  PTOs  has 
been  accomplished  to  assist  top  level  management  in  deciding 
the  appropriate  type  TO  option  to  procure.  Thus,  the 
purpose  of  this  thesis  has  been  to  analyze  the  cost  versus 
benefits  of  procuring  the  new  PTOs  for  the  F-15  series 
aircraf  t . 

To  accomplish  this  purpose,  the  following  four 
objectives  were  selected:  to  identify  the  steps  in  the  TO 
procurement  process;  to  explain  the  technique  used  by 
McDonnell-Douglas  Aircraft  Company  to  develop  cost  estimates 
for  the  F-15  weapon  system  TOs;  to  describe  the  advantages 
of  procuring  the  F-15  weapon  system  TOs  in  the  PTO  format; 
and  to  estimate  where  possible  the  potential  monetary  sav- 
ings derived  from  using  the  improved  PTO  format.  These 
objectives  were  established  so  that  it  would  be  possible 
to  systematically  provide  the  background  necessary  to 
answer  this  thesis  research  question:  Should  the  F-lt> 
ueapon  system  technical  data  be  procure.;'  in  the  FTO 
format? 

A comprehensive  literature  review  revealed  that 
the  problem  of  inadequate  TOs  was  identified  over  fifteen 
years  ago.  The  most  significant  study  of  TO  systems  since 
the  1962  study  was  completed  in  197b  by  the  Air  Force  Human 
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Resources  Laboratory  (AFHRL) . AFHRL  evaluated  three  types 
of  technical  data  for  troubleshooting.  Fully  proceduralized 
troubleshooting  aids  (FPTAs) , LTTAs,  and  CTOs  were  compared 
in  actual  performance  tests  by  apprentice  technicians, 
technicians  with  six  months  or  less  experience,  and  tech- 
nicians with  more  than  six  months  experience.  The  FPTAs  are 
not  being  considered  for  procurement  due  to  the  extremely 
high  cost  of  the  additional  pages  required  to  present 
technical  data  in  that  format. 

The  F-15  System  Project  Office  (SPO)  anticipated 
the  need  for  cost  data  for  the  various  alternatives  and 
requested  " Ball  Park"  cost  figures  from  McDonnell-Dougias 
for  the  possible  procurement  options.  This  cost  data  pro- 
vided the  basis  for  the  cost  versus  benefits  comparison. 

Many  of  the  advantages  of  PTOs  cited  by  the  various 
studies  were  theoretical  in  nature  and  unproven.  However, 
the  conclusions  of  the  AFHRL  study  of  three  types  of  tech- 
nical data  for  troubleshooting  were  backed  by  actual  con- 
trolled comparison  tests  of  CTOs  and  PTOs,  and  these 
documented  positive  aspects  of  PTOs  were  used  to  determine 
possible  maintenance  cost  savings.  Four  positive  factors 
with  statistical  relevance  were  selected  for  further 
analysis.  They  were:  performance  of  apprentice  and 
experienced  technicians;  number  of  problems  solved, - 
number  of  parts  used  incorrectly;  and  time  required  to 
troubleshoot  and  repair.  From  the  analysis  of  PTOs  three 
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savings  options  wore  developed:  Pessimist ic--3  percent; 
Most  Likely--15  percent;  and  Optimist ic--27  percent. 
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The  analogous  systems  approach  was  used  to  draw  a 
strong  analogy  between  the  C-141  avionics  subsystem  used 
in  the  AFHRL  study  and  the  F-15  avionics  subsystems. 

Seven  applicable  work  unit  codes  (WUCs),  which  make  up  the 
F-15  avionics  maintenance  cost,  were  selected  to  provide 
the  foundation  for  computing  the  potential  cost  savings. 

A Monte  Carlo  simulation  was  used  to  forecast  the  future 
F-15  total  avionics  maintenance  cost.  Three  points  (low, 
middle,  and  high)  on  the  cost  probability  distribution 
were  selected  to  reflect  the  possible  range  of  future 
avionics  maintenance  cost.  When  these  three  cost  figures 
were  placed  into  a matrix  with  the  three  anticipated 
percentage  savings  options,  a continuum  of  nine  possible 
savings  per  aircraft  per  quarter  was  developed.  Using  these 
figures  and  the  number  of  F-15  aircraft  projected  to  be  in 
the  inventory,  three  break-even  charts  were  constructed  and 
used  to  illustrate  the  forecast  points  in  the  future  where 
the  anticipated  savings  equal  the  cost  of  PTOs . Analysis 
of  the  break-even  charts  reveals  that  the  most  likely 
break-even  point  would  be  achieved  during  the  first  quarter 
of  FY  80. 
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SUMMARY  OF  FORECAST  BREAK-EVEN  POINTS 


PTO  Savings 
Options 

Forecast  Avionics  Maintenance  Cost 
Potential  (percent) 

(percent) 

Low-- 3% 

Middle — 50% 

High  97% 

Pessi- 

mistic 

3% 

FY  90-2 

FY  83-2 

FY  80-3 

Most 

Likely 

15% 

FY  82-1 

FY  80-1 

FY  79-2 

Opti- 

mistic 

27% 

FY  80-3 

FY  79-3 

FY  79-1 

1 
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illustrates  the  broad  spectrum  of  possible  break-even 
points.  The  most  likely  break-even  time  appears  to  be 
during  FY  80. 

Finally,  after  all  factors  are  considered,  the 
answer  to  the  research  question  must  be:  Thv  F~ i .S  uaapon 
system  technical  Jute  should  be  procured  in  the  PTC  (MIL-M- 
834H5J  format.  This  conclusion  is  consistent  with  the 
recent  F-15  SPO  decision  to  procure  procedurelized  tech- 
nical orders  for  tae  entire  F-15  weapon  system  (Jo). 

Keep  mme n d a t i o n s 

The  following  recommendations  aie  an  outcome  of 
this  thesis  effort. 

1.  After  sufficient,  data  is  accumulated,  the 
effect  of  PTOs  on  C-141  aircraft  maintenance  costs  should 
be  evaluated  to  determine  the  actual  savings  realized. 

The  cost  data  for  the  C-141  and  other  operational  aircraft, 
such  as  the  F-15,  A-10  and  F-lll,  which  transistion  to 
PTOs  shouLd  be  monitored  closely  in  order  to  detect  any 
positive  or  negative  trends  and  to  determine  the  causes 
of  these  changes. 

2.  Beginning  with  the  F-16  weapon  system,  all 
new  aircraft  acquisitions  will  be  procured  with  the 
procedural! zed  technical  data  package  (16).  The  mainte- 
nance cost  data  for  these  new  "state  of  the  art"  weapon 
systems  should  be  scrutinized  closely  to  detect  cost 
trends  that  can  be  attributed  to  PTOs. 

*J4 


3.  The  affect  ot  PTOs  on  those  F-1S  and  F-lt> 
aircraft  dedicated  to  foreiqn  military  sales  should  Ik* 
studied.  The  influence  ot  the  improved  capability  of 
PTOs  should  enhance  those  foreiqn  sales  and  provide  bettci 
maintenance  performance  m those  rounti  ie;i. 

■I.  The  F-15A  technical  data  has  already  been  pro- 
cured in  the  CTO  format  and  the  F-15U  could  also  bo  pro- 
cured m the  same  format  for  $0.2  million.  Consequent ly , 
an  extremely  valid  comparison  ot  the  technical  orders  could 
be  achieved  it  the  F- 1 S C 0 technical  data  were  procured 
m the  PTO  format  . It  this  comparison  verit  icd  the 
advantaqes  of  PTOs,  follow-on  procurement  ot  the  PTOs  tot 
the  F-l‘>  A/B  would  increase  the  total  cost  ot  the  technical 
data  by  only  $0.2  million.  Actually,  less  than  $200,000 
appears  to  be  a small  price  to  pay  to  validate  the  savinqa 
claimed  for  PTOs . Afterwards,  the  translation  to  PTOs 
tor  other  weapon  systems  could  bo  approached  with  qreater 
confidence  in  the  potential  of  PTOs  to  reduce  maintenance 
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MIL-il-25098 . Conventional  TO  (CTO)  procurement  specification. 
TOs  procured  under  this  military  specification  (Mil  Spec) 
contain  a description  of  the  equipment  and  most  of  the 
information  required  to  troubleshoot  the  equipment. 
However,  the  TOs  are  prepared  assuming  that  the  techni- 
cian has  adequate  training  and  experience  in  using  test, 
in  locating  most  parts  within  the  equipment,  and  in 
interpreting  schematic  diagrams  (29:92).  Some  trouble- 
shooting information  is  provided,  but  the  steps  are  not 
fully  procedural ized  (see  Appendix  B) . 

MIL-M- 38800.  The  primary  difference  between  this  specifica- 
tion and  MIL-M-25098  is  the  requirement  for  organiza- 
tional manuals  to  be  Job  Guide  Manuals  (JGMs) . These 
JGMs  are  the  new  style  Logic  Tree  Troubleshooting  Aids 
(LTTAs ) which  provide  a step-by-step  procedure  (see 
Appendix  B)  for  troubleshooting  (29:92). 

MIL-M-83495 . This  specif ication  expands  upon  MIL-M-38800 
by  requiring  Fault  Isolation  Manuals  (FIMs),  Fault 
Reporting  Manuals  ( FRMs ) , and  System  Schematics  Manuals 
(SSMs) . These  manuals  are  used  to  identify,  isolate, 
and  report  discrepancies  found  by  aircrew  members  and 
maintenance  personnel.  The  indexing  system  used  in 
these  manuals  (see  Appendix  B)  identifies  the  specific 
JGM  required  to  troubleshoot  the  malfunctioning  system. 

MI L-STD-8C3A.  The  manuals  above  are  tied  t.o  engineering 
through  MIL-STD-863A  which  provides  specific  guidance 
for  the  preparation  of  schematic  diagrams  (see  Appendix  B) 
contained  in  the  System  Schematic  Manuals  (SSMs) . 

TOMA.  A Technical  Order  Management  Agency  (TOMA)  is  an 
integral  part  of  each  Systems  Program  Office  (SPO) . 

This  agency  is  tasked  with  monitoring  the  procurement, 
validation,  and  verification  of  technical  orders  during 
the  acquisition  phase  of  new  weapon  systems. 

VALIDATION.  Contractor's  test  of  a new  TO.  Prior  to 
delivery  of  each  new  weapons  system  component,  the 
contractor  is  required  to  validate  the  accuracy  of 
each  supporting  TO  (29:92). 

VERIFICATION . Air  Force's  tdst.  of  a new  TO.  AFR  8-2 
requires  verification  of  "selected"  (but  unspeci- 
fied) TOs  prior  to  the  TOs  being  formalized  and 
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accepted  by  the 
stated  criteria 
therefore,  each 
to  ver if ication 


Air  Force.  Presently,  there  is  no 
on  what  constitutes  a verification; 

SP O develops  its  own  policy  in  reyard 
(29:1b)  . 


FPTA.  The  fully  procedural i zed  troubleshoot! ny  aid  (FPTA) 
provides  complete  step-by-step  instructions  for  both 
checkout  and  troubleshooting,  and  they  are  des iyned 
for  use  by  both  experienced  and  inexperienced  techni- 
cians. FPTAs  are  more  procedural ized  than  LTTAs,  and 
the  keyed  illustrations  provided  with  the  text  make 
the  much  more  expensive  (18:5). 

CTO.  See  MIL-H-25098. 

PTO.  As  used  in  this  thesis,  the  term  procedural i zed 

technical  orders  (PTOs)  include  those  manuals  required 
by  MIL-M- J8800,  M1L-M-83495,  and  MI b-STD-86 3A . 

UUC . Work  unit  codes  are  the  common  denominator  in  the 
I ROS  program  and  is  similar  to  the  Work  Breakdown 
Structure  (MI L-STD-88 1 ) used  by  the  prime  weapon  sys- 
tem contractor.  By  using  work  unit  coding  there  is 
a complete  identification  from  the  major  functions  of 
the  system  (landing  gear,  bomb/NAV,  etc.)  down  to  its 
lowest  level  component  (27:8). 

HOC.  The  Required  Operational  Capbility  (ROC)  is  now 
referred  to  as  a Statement  of  Operation  Need  (SON) 
and  at  one  time  was  referred  to  as  a General  Opera- 
tional Requirement  (GOK) . It  is  the  primary  vehicle 
for  stating  a Major  Air  Command  need  (17:9). 

PROGRAM  TRANSFER . The  point  in  time  when  the  responsi- 
bility for  a weapon  system  is  transfei red  from  the  Air 
Force  Systems  Command  to  the  Air  Force  Logistics 
Command . 
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APPENDIX  B 


TECHNICAL  ORDER  COMPARISON 
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r.o.  5ru- 3 is  3 


cmcKOui  Asn  irourlimioot  control 
INDICATOR  C-3«l9A/A‘.N-3i  (COfIT) 

Inspect  Control  Indicator. 

CAUTION 

If  returning  to  this  page  from 
a troubleshooting  step,  be  sure: 

a.  All  disconnected  wires 
and  cables  are  recon- 
nected. 

b.  All  removed  components 
are  reinstalled. 

I.  Open  foldout  (p.  1 59). 

NOTE 

Oo  not  overlook  importance  of 
visual  inspections.  If  you  do 
not  know  how  to  inspect  com- 
ponents, refer  to  T.O.  MS1M 
GENL. 

Replace  defective  components, 
as  discovered,  and  continue 
checkout.  Refer  to  IPB  (T.O. 
5N5-12-2-4)  for  part  number. 


Chei!  Counter  Assemblies. 

NOTE 

Re  sure  to  rotate  L/R  counter 
assembly  (G)  through  its  full 
count  999  LEFT  and  RIGH1  999. 

A.  Operate  controls  (/),  (8),  and 
(11)  to  rotate  counter  assemblies 
(G),  (9),  and  (5)  through  their 
full  count.  Check  that  counters 
(6),  (9),  and  (5)  rotate  freely. 

If  not,  inform  supervisor. 

Check  Torque  Transmi tter  ^ontinui ty 

5.  Disconnect  P5405  (15).  Using  VOM, 
check  that  resistance  between 
P5401-30  (3)  and~P540 1 - 32  (4)  is 

8 to“1 6_ohms . if  not',  go  to 
p.  1-1?  (malfunction  No.  1). 

6.  Connect  P5405  (15).  Disconnect 
P5406  (16). 

7.  Using  VOM,  check  that  resistance 
between  P5401-30  (3)  and  P5401-32 
(4)  is  8 to  lu  ohms.  If  nut,  go 
to  p.  1-12  (malfunction  No.  2). 


3. 


P erform  tho rough  visua  1 i nspe c t i ojn 
of  control  indicator  ( 1 2) . Look' 
for: 

a.  Burned  or  scorched  components. 

b.  Cracked,  disconnected,  and 
broken  wires. 

c.  Bent,  broken,  and  missing  pins 
(13)  and  terminals  (14). 

Inspect  control  indicator  (12)  for 
foreign  matter.  Remove  foreign 
matter,  as  required. 


TYPICAL 

14- 


ROTAltO  90° 


TYPICA1 

13 


Example  Pago  from  FPTA  Checkout  and  Troubleshoot:  Procedures  for 
Navigational  Computer  AN/ASN-H5 
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1.0.  !>NI  -3-TS-3 


LHtCKCUl  AitU  1 RUUUI  L SH001  CUNIROl 
INOICAIOR  C-3U19A/ASN  >b  (fO.NI) 

Troubl >■  shoot  Hal fu nt t i on  Ho,  3/  (Cont). 

4.  Chock  that  right  counter  magnet 
assembly  K5-J0S  (4)  adjustment 
is  good.  ~Tf  not , do  to  step  11. 

If  adjustment  is  yood,  yo  to  nest 

steo. 

b.  Check  that  riuht  counter  magnet 
core  KS40b  (3)  adjustment  is  good. 
If  not,  qo  to  step  12.  If  adjust- 
ment is  yood,  go  to  next  step. 

6.  Inform  supervisor  malfunction  is 
mechanical  and  located  in  l/H  alonq 
track  counter  assembly  (2). 

7.  Using  VOM,  check  that  resistance 
between  SViS402  CIO  (b)  and 
SVjb402-ClV  (6)  i s~l  ohm  or  less. 

Tf  not",  qo'  to  step"Tj.  Tri/WT 
indicates  1.  ohm  or  less,  qo  to  next 

step. 

8.  Disconnect  PS406  (8)  and  go  to  next 

step. 


9.  IJ.iny  VtVS,  check  that  resistance 
between  J5406-J  (10)  a n'iTT!>4 06- f 
(9)  is  hiY  to  DO  ohms,  if  Mt,  qo 
to  stcpTC  1 T V 0M~  i no icates  60 
to  80  ohms,  go  to  next  step. 

10.  Using  control  indicator  wiring  dia 
gram  (vol.  6)  check  wiring.  Repair 
or  replace  defective  wires  and  go 
to  p.  1 -1 . 

11.  Adjust  right  counter  magnet  assem- 
bly KblOb  (4)  (vol.  6,  p.  S-I) 
and  qo  to  step  lb. 


1-42 

Example  Page  from  1'PTA  Checkout  and  Trouble  sho.it  Pi  ooodm  os  tot 
Navigational  Computer  AN/ASN-ii 
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T.O.  5N \ -3-8-2-TS- 1 


Fiom  blw|>  1-Vi 


J.  Disconnect  audio  oscillator 
from  tost  harness. 

2.  On  ti  st  harness,  monitor  ac 
voltage  between  test  points 
A S and  An,  usj/ty 
Flt^ct  ronic  Voltmeter. 

?.  On  control  indicator , set 
STAGE  J DISTANCE  TO  GO 
readout  /list  to  000  N.M. 
then  to  5 00  N.M. 


On  te*t  harness,  are  ac  nulls 
I ics«  nt  between  test  points  AS 
and  A<»  when  STA..F  I DISTANCE  IT) 
» .O  iCAilout  on  control  indicator 
»s  set  to  OOO  N.M.  and  f»00  N.M. 


An  navigation  multiple 
indicator  faulty. 


t\ult  *pl'* 

f- 


Control  indicator  faulty. 
Perform  control  indicator 
bench  checkout  procedure. 


figure  3-3.  Computer  Set  Troubleshooting  for  failed  Bench 
Checkout  Procedure  3-5/3 


Example  Page  from  ETTA  Bench  Checkout,  and  Troubleshooting 
Procedures  for  Computer  Sot  AN/ASN-35  (Section  3 of  1.T1A) 
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k A I i r . ' 


i 


1 M'U  A . . ON 


l‘Kv  v'l  .k  .*•: 


' - ■»  * I Oa  a 1 1 navuj.it  ion  mdi 

j calm  an  l eontiol  uW.icat  'f, 

! OlSTANd:  TO  i»i»  U’.Klmit  n ..  * not 
agree. 


3 - S 


3- 


s *s 


On  control  uuUiMtor  , STAG1;  ll 
l)l,STANi't  TO  GO  i eadout  nut  de 
or eas i ng . 

i'u  a\ii  navigation  multiple  iruii 
ratot  . PISrANGE  TO  GO  readout  not 
deereab  mg  . 


3-5/b  On  control  indicator  and  an  navi 
gation  multiple  indicator, 

01 STANCH  TO  GO  readouts  do  not 
agree. 


3-S  / On  rontiol  indicator,  STAG!  I l 
DISTANCE  TO  GO  UMilout,  not  in 
crcds  mg . 


3-*>  8 ‘ On  control  nuluatai  , STAGE  l 

j PIS  lANdf  ’i'O  GO  readout  not  in- 
creasing. 


it  all  %u  l o l t art: 

; ir  .rat  and  va r / » air 
: r it  mu  < . I t i ; • l e iri.il 
ca  t t»i  i *.  : iu  lt\  . Other  - 
wise,  con  t ; e l i \u  l ca  t Oi 
is  faulty  iv  i • orir  con- 
trol i nd  i «.  .» t ot  ii.TH'h 
c tn  v‘ sou  t \ : octs.  .re, 

Figut'v  i t. 


v'ontiol  m.luMioi  faulty  . 
Pe 1 t’ orm  cvntiol  md  u ,ito. 
Ihmu  h checkout  pi  ocedut e . 

iVnttol  uuluu’at  faulty. 
Perform  rant  tol  mdicat  > r 
bench  checkout  p:oct*duie. 

do  it  t *1  indicator  faulty  . 
Pc  i l ot  rantiol  i nd  uwtoi 


bench 

c hi*c 

‘.oat  procedure. 

On  ( e 

st  harness , 

cheek 

1 O'l  .< 

\ 

tv.  15 

V positivi 

}ui  1 '.o> 

• » s - 

to  8s 

ms.  cl  ura- 

t  l(M. 

' »CC  U l 

ring  *3 

te  1 1 

t i me 

per 

mnut  e 

at  te.t 

pe  i n t 

••  ft* 

l hot  ) 

anil  US 

it  tn) 

I 1 

pu  l sc 

is  pres- 

nut  . 

.'nut  : 

o l i nd 

ieat  or 

is  faulty. 

Ot  he* 

i .vise. 

iuvu.it  ional  computer 
ti.iv  k resolver  drive  300 
is  faulty.  Perform  track 
rcsolvci  drive  bench 
choc  ‘.on  t pi  ocedut  e . 

v’ontiol  indicator  taalty 
IVi  oi.\  coittoi  indicator 
bench  checkout  I'nvoJtuv. 


i\unpU*  l\i£o  t i o.a  l.TTA  Bench  Checkout  and  iiruhlr  .h.ut  in*’, 
Procedures  for  Computet  Si' l AN/ASN  V> 


f Indication  If  Indication 

Is  Normal  Is  Abnormal 


Section  VIII 


Figure  b-2.  T > of  Control- Ir.dica'.ur  Trouble  Analysis  (Sheet  3 of  G) 


= ~“s>  - -aaa - rr  . -gr-s. 


MONTE  CARLO  PROGRAM 


001*?P,UN  *=  (ULIU) CD ADLIB /TSS,R ; LIBRARY /AP PL IB ,K  ; 

002*»OCMK.LIB/LIBRAKY,R;AF.LIB/BLIBRARY,R 

010  DIMENSION  Y (5000) 

020  CALL  ATTACH  (10,  "78A71 /CET1A; ", 3, 0) 

030  00  20  J-1,4000 
035C 

040  X1-UNIF0RM(2S2. ,870.) 

050  X2-LOGNORM(486.25,407.182) 

060  X3=*LOCNORM(2  74.0, 382. 102) 

070  X4=UNIFORU(563.  ,1516.) 

080  X5-XNORMALC940. ,279.) 

090  X6-L0GNORM(2245. 75,966.597) 

095  X7-EXPONT( 16332.) 

09  7C 

100  Y(J)  = X1+X2+X3+X4+X5+X6+X7 
110  WRITE ( 10, 105 ) J,  Y(J ) 

120  105FORMAT(I4,2X, FI  2. 1) 

125  20  CONTINUE 
130  F.NDFILC  10 
140  REWIND  10 

150  STOP  "FILE  10  IS  COMPLETE" 

160  END 

170  FUNCTION  LOGNORM(U, SO) 

180  LOCNORM-EXP (XNORMAL ( ALOC (U ) - . 5*ALOG ( ( SD /U ) **2+1 . ) , 
190&  ALOG( (SD/U) **2+i . ) ** . 5) ) 

200  RETURN 
210  END 
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SAMPLE 

OF  DATA  FROM 

MONTE 

CARLO  PROGRAM 

1 

13142.0 

'> 

201 79.7 

J 

22030.2 

4 

65977.7 

5 

0884. 0 

f. 

53411.1 

7 

15513.3 

3 

9276.7 

0 

29626.4 

10 

45057.7 

11 

13311.8 

12 

5555.5 

13 

63970.9 

14 

1 5667.1 

IS 

36895.7 

IS 

7713.0 

1 7 

39432 . 7 

IS 

2 382°. 3 

10 

22702.  l 

20 

20111.8 

21 

17521.0 

22 

35065.0 

23 

8286. 8 

24 

34  50 . 4 

25 

20028.5 

26 

46092.0 

27 

21623.0 

23 

26365.0 

29 

15349.0 

30 

19820.5 

31 

1 1864.2 

32 

14873.3 

33 

1,3334.3 

34 

8980.2 

35 

24333.9 

l 
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PROBABILITY  DISTRIBUTION  PROGRAM 


I, 


i.  » i.*  A 


T 


IO##NORH,S(SL) 

20$  : T DENT:WP1 308, AFIT/LSG(MORAVEK  STD  78A71  WP1308)  CRAPH01  PLOT 
30$:MSG2:  L.SEND  PLOT  TAPE  TO  PLOTTER**USE  BLACK  INK**(MORAVEK,WP1308) 
40$ : OPT  ION : FORTRAN . NOMAP 
50$: FORTY: NFORM.NLNO 
60  DIMENSION  DAT (5000) 

70C 

80C  PROGRAM  (PLOTFREQ)  FOR  DATA  (DAT(K))  SCALED  (LINE  430) 

900  BETWEEN  ZERO  AND  TEN.  PARAMETER  IV  - 1 GIVES 

LOOC  PROBABILITIES;  IV  - 0 FREQUENCIES  (CHANGE  LABELS) 

105C  PARAMETER  JK  - 1 GIVES  3 BY  5 PLOT;  5 BY  5 OTHERWISE. 

110C  PROGRAM  SKIPS  LINK  NUMBERS;  XO  IN  READ  IS 

1 20C  VALUE  FOLLOWING  LINE  NUMBER;  CHANGE  READ  LIST 
1 30C  FOR  CONFORMITY  TO  FILE  BEING  READ. 

1 35C  CHANGE  XMIN.XMAX  AS  DESIRED 
1 36C 

140  PARAMETER  IV  » 1;  JK-1;  XMAX  - 20;  XMIN  = 0. 

150  REAL  YVAL( 200 ) 

160  CHARACTER  OAUSE*l,  ALPHA* 1 

170  CHARACTER  TITLE1*34/'  DISTRIBUTION  OF  COMPUTED  V / 

180  CHARACTER  TITLE2*37/'  AVIONICS  COST-TOTAL  V/ 

190  K-l 

200  TLO >*999999. 

210  TUI— 999999. 

220  SUM-0. 

230  SQU-0. 

240  5;iEAD(5,  1001,  END-50)  XO 
250  DAT(K)  - XO 
260  1001  FORMAT (V) 

270  1F(0AT(K) .GE.THI)  THI-DAT(K) 

2 BO  1 F ( DAT ( K ) . LE. TLO ) TLO-DAT(K) 

290  SUM-SUM+DAT (K) 

300  SQU  = SQL!  + DAT (K) **2 
310  K-K+l 
320  GO  TO  5 
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330  SO  CALI.  (JSTART 

3 AO  CALL  Ul)IMEN(8.  ,33.  ,"CAPT  MORAVKK  78-A\") 

3 SO  K-K-l 

360  CALL  1)  SET  ("SMALL") 

370  CALL  UDAR EA(0.,12.,0.,12.) 

3 SO  Y » K 

3^0  DO  100  J =*  1,30 
400  lOOYVAL(J)  » 0 
410  YMAX  =■  0. 

420  DO  200  J - 1,K 

430  INDX  - DAT(J)/5000.  + 1 

440  IK  = MAXI (IK, INDX) 

450  YVAL(INDX)  - YVAL(INDX)  + l 

460  IF(YVAL(1NDX) .GT.YMAX)  YMAX  - YVAL(INDX) 

470  200CONTINUE 

480  YMAX  - (YMAX/ 10.  6 l.)*10. 

490  AVE  - SUM/Y 

500  IF  (IV.NF..1)  GO  TO  206 

510  DO  107  M - 1 , I K 

520  YVAL(M)  - YVAL(M) /Y 

523  PRINT  999,  YVAL(M) 

527  099  FORMAT ( IX, F6. 3) 

530  107CONTINUE 
540  YMAX  - YMAX/Y 

550  206PRINT  1011,  (DAT(M) ,M“l , IK) , YMAX,  FLOAT (INDX) 
560  101 1F0RMAT(3X,  10F6.2) 

5 7 0 S I CM A-S QR T ( ( S QU-( S UM* * 2 ) / Y ) / ( Y - 1 ) ) 

580  7MED“ (TLO+THI) /2 • 

590  CALL  1ISET("DASH") 

600  CALL  OWINDO(0.,100.,0.,100.) 

610  CALL  DMOVE(0. ,0. ) 

620  CALL  URECT(100. ,100.) 

630  CALL  US ET("LINE") 

640  IF(JK.EQ. 1)CALL  UDAREA(2.0,  7.0,  3.0,  6.0) 

650  IF(JK.NE.  UCALL  UDAREA(2 .0, 10. , 3.  , 8. ) 
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660  CALL  UPSET("YLABEL", "PROBABILITIES  \") 
670  CALL  HI’S l'.T ( "Xl.ABEL"  , "COST  TIMES  5000V) 
oKO  CALL  USET  ( "XBOTHLABELS" ) 

600  CALL  U S KT  ( "Y  BOT  H LA  B ELS  " ) 

700  CALL  IISET("0WNSCALE") 

710  CALL  UV l NDO  ( XM I X , XMAX , 0 • , YMAX ) 

720  CALL  1)AXIS(XMIN,XMAX,0. , YMAX) 

730  00  300  J - 1,30 

740  X - J -1 

750  CALL  UMOVE(X,0.) 

760  CALL  URECT(X+1 , YVAL(J ) ) 

770  300CONTINUF. 

780  CALL  IJSET( "DEVICE") 

790  CALL  OMO VE (2.25,9.85) 

800  CALL  UPRNT1  (TITLE  I, "TEXT") 

810  CALL  UMOVE(2.25,9.85) 

320  CALL  UD01T ("Ll'Ol") 

330  CALL  Ul’RNT  1 (T ITLE2  , "TEXT") 

860  CALL  UMOVE (5.9,9.25) 

S50  CALL  UPRNT1  ("MEAN  -\","TEXT") 

860  IF(Y.NE.O.)  GO  TO  250 

370  CALL  UPR NT  1 ( "NO  DATA\ " , "TEXT" ) 

8S0  CO  TO  490 

890  250  CALL  UPP.NT1 (AVE, "HEAL") 

900  CALL  UMOVE (5. 9, 9. 25) 

910  CALL  UDOIT  ("l.FOi") 

920  CALL  UPRNTIC’ST  l>EV-\","TLXT") 

930  IF(Y.EQ. 1.)  SIGMA-0. 

940  CALL  UPKNT1 (SICMA,"REAL") 

950  CALL  UMOVE (5.9,9.25) 

960  CALL  UDOIT ("LF02") 

970  CALL  UPRNT1 (Y,"XEAL") 

930  CALL  UPRNT1  (T'1ED,"REAL") 

990  CALL  UMOVE (5. 9. 9. 2 5) 

1000  CALL  UDOIT ("LF03") 


i 


i 
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1 HO  CALL  UPRNT1  ("ill  PCT-\","TEXT") 

1020  CALL  1JPRNT  i (TO I , "R  KAL") 

H30  CALL  UMOVr(5.<1,0.25) 

1040  CALL  UDOIT("LF04") 

IO'jO  CALL  UPRNT1 ("LO  PC T-\" ."TEXT") 

10SO  CALL  UPRNT 1 (TLO , "REAL" ) 

1070  CALL  UMO  VE (1.05,9.45) 

1080  CALL  UPRNT  I ("BKNNETT/flOKAVEK\", "TEXT") 
1000  400  CALL  US ET ( "VIRTUAL") 

HOC  500  CONTINUE 
1110  CALL  UENO 
1120  STOP 
11)0  END 

1 140$ '.LIBRARY : Al , A2 , A3 
1 150$: EXECUTE 
1 160$ : LIMITS : , 30K 

1 170$: PRMKL: A 1 ,K , K, GRAPH  ICS. LIH/GCS/GCS3 . 0 
1 1 80$ : PR  MEL : A2 , R , R , CR  API1I C S . LI  B /CC  S/C  ALL  ) . 0 
1100$: PRMFL: A3 , K , K , AE . LIB/ CALL IB 
1200$:  KFII.E:  27,  FIXLNG/RO,  ttUFSIZ/81 
1 210$ : TAPE: 2 7, XT' , , , , PLOT-TAPE/ WR 
1220$: DATA: I* 

1 2 30$ : S ELEC  1’A : 78A7  l/C ET  1 A , 11 
1240$: ENOJOB 


..  . ... 


SIMFIT  PROGRAMS — SAMPLE 


010#*S,R(SL)  : , T,  15; ; ,61 

020$:  I0r.NT:WP  1308,  AFIT/MORAVEK  7RA/LSG 

030$: or:  LON : FORTRAN , NOMAP 

040$: SCIXCT: SI  (FIT/SFO-nECK 

050$: EXECUTE 

0f>0$: LIMITS: 05, , ,5000 

070$:0ATA:05 

0S0$:SELFXTA:SF1 

090$: P\T\: 09 

100$:SELF.CTA:WUC1 

1-10$:  UNO  JOB 


10  1 2 3 4 5 6 7 S 9 10  11  12  0 0 0 

020  .104  1 00 

030  .75  282.0  163.3333  4 0.0  0 0 0 0.0 
C 40  TEST  OF  WHC  5 1 XXX 
050  TEST  NO.  I 
060  ; 1 
073  (V) 

080  -99.99 


010  583  370  767  679 

020  730  232  530  500 

030  550  485  535  492 

040  -39.99  00  00  90 
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